Skip to content

User Interface

The ExplorerUI application and its building blocks.

lumen.ai.ui

CODE_EXEC_WARNING = '\n### ⚠️ Security Warning\n\nEnabling code execution allows the AI to generate and run Python code on your system.\n\n**This poses significant security risks**, especially in environments with:\n\n- API keys or secrets\n- Database credentials\n- Access to sensitive files or systems\n- Network access to internal services\n\nLLM-generated code may be unpredictable. While validation layers reduce accidental errors, they **do not provide meaningful protection** against malicious or compromised inputs.\n' module-attribute

DATA_SOURCES_HELP = '\n**Add data to explore with Lumen AI.**\n\n**Supported formats:**\n\n- CSV, Parquet, JSON, Excel, GeoJSON files\n- Remote URLs (https://example.com/data.csv)\n- Databases via SQLAlchemy\n\n**How to add data:**\n\n1. **Upload Data** — Drag and drop files or click to browse\n2. **Fetch Remote Data** — Enter URLs to fetch remote data\n\n**Tips:**\n\n- Upload .md or .txt files alongside data to give the AI context\n- Multiple sources can be used together in queries\n- Data connections persist for your session\n' module-attribute

DEMO_MESSAGES = ['What datasets are available?', 'Show me the the first dataset and its columns.', 'What are some interesting analyses I can do?', 'Perform an analysis using the first suggestion.', 'Show me a plot of these results.'] module-attribute

DataT = str | Path | Source | Pipeline module-attribute

EXPLORATIONS_INTRO_HELP = '**Select a table below** to start a new exploration. Click on Help on the left for more info.' module-attribute

EXPLORATION_CAPTION = '\nLaunch new explorations by chatting or selecting a table.\n\nAsk follow-up questions by navigating to an existing exploration.\n' module-attribute

EXPLORATION_TOOLBAR_HELP = '\nToolbar actions for this exploration:\n\n- **Export Output** — Download as CSV/Excel (tables) or PNG/SVG (visualizations)\n- **Copy** — Copy the spec to clipboard\n- **Sparkle** — Ask AI to revise the output\n- **Annotations** — Add highlights to visualizations (plots only)\n- **Pop-out** — Move to a separate pane\n\nDrag the divider between editor and view to resize.\n' module-attribute

EXPLORATION_VIEW_HELP = 'Use < > to expand/collapse panels. Edit the spec (top) and the view (bottom) syncs. Click ✨ to ask LLM to revise.' module-attribute

GETTING_STARTED_SUGGESTIONS = [('search', 'What data is available?'), ('description', 'Show me a dataset'), ('bar_chart', 'Visualize the data')] module-attribute

HELP_EDITOR = "\n**📝 Two-pane editor:**\n\nWhen you ask a question, you'll see two panes:\n\n- **Top pane** = the spec (SQL query or YAML configuration)\n - Edit directly here to modify the analysis\n - Changes are applied in real-time to the preview below\n - Great for small tweaks like changing colors or sort order\n\n- **Bottom pane** = live preview\n - Automatically updates when you change the spec\n - Shows tables, charts, or other visualizations\n" module-attribute

HELP_EXPLORATIONS = '\n**🔍 What are Explorations?**\n\nAn Exploration is a persistent, contextual workspace for working with a specific dataset. It is created when a SQL query is generated and captures the full state of the interaction, including the conversation, generated analyses, visualizations, and other data artifacts.\n\nExplorations can evolve over time, support multiple questions on the same data, and can be revisited or exported as a coherent unit.\n\nExplorations start from the global context (available sources and metadata). If a question is a follow-up, the new exploration is nested under the parent; if it is not, Lumen creates a new top-level exploration.\n\n**🧭 Navigation menu (left panel):**\n- Shows all explorations in a persistent tree\n- Click an exploration to switch context\n- Follow-up explorations appear nested under their parent\n- Use the item menu (⋮) to remove an exploration\n' module-attribute

HELP_EXPORT = '\n**📄 Export individual results:**\n\n- **Export Output** ▼ (top of each result) - Download specific results:\n - **Tables**: SQL, CSV, or Excel formats\n - **Visualizations**: PNG, JPEG, PDF, SVG, or HTML\n\n**💾 Save your work:**\n\n- Use **Export Notebook** in the navigation menu to download everything as a Jupyter notebook\n- Includes all SQL queries, visualizations, and results\n- Perfect for documentation or sharing with colleagues\n- Each exploration saves automatically during your session\n\n**📋 Report Mode:**\n\n- Click **Report** in the left sidebar\n- View all explorations on one page\n- Export everything as a single notebook\n' module-attribute

HELP_GETTING_STARTED = '\nAsk questions in plain English and Lumen AI automatically generates SQL queries and creates visualizations.\n\n**🚀 Start exploring:**\n\n- Type a question like "What datasets are available?"\n- Click the quick action buttons below the chat\n- Select a table and click **Explore** to start analyzing\n\n**❓ Example questions:**\n\n**Basic queries:**\n- "Show me the top 10 rows"\n- "What are the columns in the dataset?"\n- "Give me a summary of the data"\n\n**Visualizations:**\n- "What\'s the average revenue by region? Plot as a bar chart"\n- "Create a scatter plot of X vs Y, colored by category"\n- "Show me a histogram of sales with 20 bins"\n\n**Complex queries:**\n- "Filter for records over $1000 and show the distribution by category"\n- "Group by department and calculate average salary. Then plot the result."\n- "Are there any outliers in the data?"\n' module-attribute

HELP_INTERFACE = '\n**🖥️ Panel Controls:**\n- **< >** buttons - Collapse/expand the chat and results panels\n- **^ v** arrows - Expand/collapse individual analyses\n\n**📂 Sidebar:**\n- **Explore** - Main chat and analysis mode (default)\n- **Report** - View all explorations on one page\n- **Sources** - Add or manage your data\n- **Settings** - Control AI behavior settings and LLM configuration\n- **Help** - Open this help dialog\n\n**⌨️ Keyboard Shortcuts:**\n- **Enter** - Send message\n- **Shift + Enter** - New line in chat\n' module-attribute

HELP_NO_SOURCES = '**Welcome to Lumen AI!**\n\n⚠️ **No data sources connected yet.**\n\n**To get started:**\n\n1. Click **Data Sources** in the left sidebar\n2. Upload a file (CSV, Parquet, JSON, Excel, GeoJSON) or connect to a database\n3. Once your data is loaded, come back here to start asking questions!\n\nYou can also drag & drop files directly into the chat interface.\n\n**Once you have data, you can ask questions like:**\n\n*Basic queries:*\n- "Show me the top 10 rows"\n- "What are the columns in the dataset?"\n- "Give me a summary of the data"\n\n*Visualizations:*\n- "What\'s the average revenue by region? Plot as a bar chart"\n- "Create a scatter plot of X vs Y, colored by category"\n\n*Complex queries:*\n- "Filter for records over $1000 and show the distribution by category"\n- "Group by department and calculate average salary. Then plot the result."\n\n---\n\n➡️ After adding data, click **Navigation** above to learn how to move around the interface.' module-attribute

HELP_RESULTS = '\n**🛠️ Toolbar actions (top of each result):**\n\n- **📋** - Copy YAML spec to clipboard\n- **✨** - Ask the LLM to revise this analysis\n- **💬** - Add highlights or callouts to visualizations (plots only)\n- **↗** - Pop out into a separate pane for comparison\n\n**📊 Table features:**\n- Click **column headers** to sort (^ = ascending, v = descending)\n- Use **pagination** (< >, page numbers) at the bottom to navigate large datasets\n- All data is interactive - you can explore it directly\n\n**🔄 Refining results:**\n- **Rerun** (⟲ in the chat input + menu) - Re-execute the last query if there was an error\n- **Undo** (↶ in the chat input + menu) - Remove the last response and try again\n- **Continue the conversation** - Send a new message to refine: "Can you make that chart show only the top 5?"\n- **Manual editing** - Edit the spec directly for precise control\n' module-attribute

HELP_TIPS = '\n**🔗 Combine multiple requests:**\nYou can ask the AI to perform several steps at once.\n\n- "Filter the data for 2023, create a bar chart of sales by region, and then summarize the top 3 regions."\n\n**💡 Ask for suggestions:**\nNot sure where to start? Ask:\n- "What could be interesting to explore in this data?"\n- "Give me 3 ideas for visualizations."\n\n**✨ Refining results:**\nYou don\'t have to get it right the first time.\n- "Can you make that chart show only the top 5?"\n- "Change the color of the bars to green."\n- "Add a trend line to the scatter plot."\n\n**🗂️ Need more data?**\nClick **Sources** in the left sidebar to add files or databases at any time.\n\n**🧠 Chain of Thought:**\nEnable **Chain of Thought** in Settings to see the AI\'s step-by-step reasoning.\n- This is great for debugging or understanding complex queries.\n\n---\n\n✔️ You\'re all set! Close this dialog and start exploring.\n' module-attribute

IMAGE_MIME_TYPES = {'.png': 'image/png', '.jpg': 'image/jpeg', '.jpeg': 'image/jpeg', '.gif': 'image/gif', '.webp': 'image/webp', '.svg': 'image/svg+xml', '.bmp': 'image/bmp'} module-attribute

PAGE_SX = {'.sidebar': {'transition': 'width 0.2s ease-in-out'}, '.sidebar:hover': {'width': '140px', 'transitionDelay': '0.5s'}, '&.mui-light .sidebar': {'bgcolor': 'var(--mui-palette-grey-50)'}} module-attribute

PREFERENCES_HELP = "\n**Control how Lumen AI analyzes your questions.**\n\n**Chain of Thought** — Show the AI's reasoning steps as it works\n\n- Useful for understanding how the AI reached an answer\n- Displays expandable cards with step-by-step logic\n- Disabled by default\n\n**SQL Planning** — Run discovery queries before generating final SQL\n\n- Improves accuracy for complex questions\n- The AI explores your schema before querying\n- Disabled by default\n\n**Validation Step** — Double-check if the response answered your question\n\n- Catches data errors early\n- Verifies results match your intent\n- Enabled by default\n\n**Configure AI Models** — Choose which LLM provider and models to use for different tasks\n" module-attribute

PROVIDED_SOURCE_NAME = 'ProvidedSource00000' module-attribute

REPORT_CAPTION = 'Use ▶ to execute all, × to clear outputs, ∨∧ to collapse/expand sections. Click exploration names to jump to them.' module-attribute

REPORT_HELP = '\n**Report Mode: View All Your Work**\n\nReport mode displays all explorations in one scrollable page for review and presentation.\n\n**Page Controls:**\n\n- **▶** (Execute Report) - Run all analyses and generate fresh results\n- **×** (Clear outputs) - Remove all output displays to start fresh\n- **∨ ∧** (Collapse/Expand) - Show or hide all analysis sections at once\n\n**Navigation:**\n\n- Click an exploration name in the left sidebar to scroll to it\n- All analyses appear in order with their results\n- Scroll through the entire report or jump to specific sections\n\n**Export:**\n\n- Use **Export Notebook** in the navigation menu to download everything as one Jupyter notebook\n- Includes all SQL queries, visualizations, and results\n- Perfect for sharing findings or creating documentation\n\n**To get back:**\n\n- Click **Explore** in the left sidebar to return to chat mode\n- Or click a specific exploration in the navigation menu to work on it\n' module-attribute

SOURCE_TABLE_SEPARATOR = ' ⦙ ' module-attribute

SPLASH_HELP_HINT = 'Click on Help on the left for more info.' module-attribute

TContext = Mapping[str, Any] module-attribute

XARRAY_EXTENSIONS = ('nc', 'nc4', 'netcdf', 'h5', 'hdf5', 'he5', 'zarr', 'grib', 'grib2', 'grb', 'grb2') module-attribute

log = getLogger(__name__) module-attribute

ActorTask

Bases: ExecutableTask

An ActorTask wraps an Actor in a way that allows persisting the message history, instructions, and context as well as making it possible to track its outputs.

actor = param.ClassSelector(class_=Actor) class-attribute instance-attribute

input_schema property

level = 3 class-attribute instance-attribute

output_schema property

editor(show_title=True)

Returns the editor for the tasks.

Arguments

show_title: bool Whether to show the title of the tasks.

Returns:

Type Description
The editor for the tasks.

prepare(context) async

AnalysisAgent

Bases: BaseLumenAgent

analyses = param.List([]) class-attribute instance-attribute

conditions = param.List(default=['Use for custom analysis, advanced analytics, or domain-specific methods', "Use when the user query matches one of the available analyses' name or description below.", "Include the selected analysis' required cols in the instructions", 'NOT for simple queries or basic visualizations']) class-attribute instance-attribute

input_schema = AnalysisInputs class-attribute instance-attribute

output_schema = AnalysisOutputs class-attribute instance-attribute

prompts = param.Dict(default={'main': {'template': PROMPTS_DIR / 'AnalysisAgent' / 'main.jinja2', 'response_model': make_analysis_model}}) class-attribute instance-attribute

purpose = param.String(default='Perform custom analyses that are reliable and repeatable.') class-attribute instance-attribute

user = param.String(default='Analysis') class-attribute instance-attribute

respond(messages, context, step_title=None) async

AnalysisOutput

Bases: LumenEditor

analysis = param.ClassSelector(class_=Analysis) class-attribute instance-attribute

context = param.Dict() class-attribute instance-attribute

pipeline = param.ClassSelector(class_=Pipeline) class-attribute instance-attribute

render_context() async

BaseCodeAgent

Bases: BaseViewAgent

Base class for view agents that can generate and execute code.

Subclasses must: - Set _executor_class to the appropriate CodeExecutor subclass - Implement _generate_code_spec() for their specific code generation flow

code_execution = param.Selector(default='disabled', objects=['disabled', 'prompt', 'llm', 'allow'], doc="\n Code execution mode for generating visualizations via code:\n - disabled: No code execution; generate declarative specs only (safe for production)\n - prompt: Generate code, prompt user for permission to execute\n - llm: Generate code, validate with LLM safety check, then execute\n - allow: Generate and execute code without user confirmation\n\n ⚠️ WARNING: The 'prompt', 'llm', and 'allow' modes execute LLM-generated code and\n must NEVER be enabled in production environments with access to secrets, credentials,\n or sensitive data.\n ", allow_refs=True) class-attribute instance-attribute

code_execution_enabled property

Whether code execution is enabled (any mode except 'disabled').

BaseSourceControls

Bases: Viewer

Base class for source controls with lifecycle management.

Provides progress reporting, message display, load-button wiring, and source-output registration. Subclasses implement _load() or call _run_load() manually for custom trigger patterns.

cancel = param.Event(doc='Cancel') class-attribute instance-attribute

context = param.Dict(default={}) class-attribute instance-attribute

disabled = param.Boolean(default=False, doc='Disable controls') class-attribute instance-attribute

label = param.String(default='', constant=True, doc='\n HTML label shown in the sidebar for this control.') class-attribute instance-attribute

load = param.Event(doc='Trigger data loading') class-attribute instance-attribute

load_button_icon = param.String(default='download', doc='\n Material icon name for the load button.') class-attribute instance-attribute

load_button_label = param.String(default='Load Data', doc='\n Text label for the load button.') class-attribute instance-attribute

load_mode = param.Selector(default='button', objects=['button', 'manual'], constant=True, doc='\n How data loading is triggered:\n - "button": Show load button, clicking triggers _load()\n - "manual": No button, use _run_load(coro) for custom triggers\n ') class-attribute instance-attribute

outputs = param.Dict(default={}) class-attribute instance-attribute

source_catalog = param.Parameter(default=None, doc='Reference to SourceCatalog instance') class-attribute instance-attribute

source_name_prefix = param.String(default='UploadedSource', doc='\n Prefix for auto-generated source names.') class-attribute instance-attribute

upload_successful = param.Event(doc='Triggered when files are successfully uploaded and processed') class-attribute instance-attribute

ChatAgent

Bases: Agent

ChatAgent provides general information about available data and other topics to the user. When data is available, it acts as an analyst providing insights and interpretations.

conditions = param.List(default=["Use for general conversation that doesn't require fetching or querying data", 'Use for technical questions about programming, functions, methods, libraries, or APIs', "Use when user asks to 'explain', 'interpret', 'analyze', 'summarize', or 'comment on' existing data in context", "NOT when user asks to 'query', 'filter', 'calculate', 'aggregate', or 'transform' data", 'NOT for creating new data transformations - only for explaining data that already exists']) class-attribute instance-attribute

output_schema = ChatOutputs class-attribute instance-attribute

prompts = param.Dict(default={'main': {'template': PROMPTS_DIR / 'ChatAgent' / 'main.jinja2'}}) class-attribute instance-attribute

purpose = param.String(default='\n Provides conversational assistance and interprets existing results.\n Handles general questions, technical documentation, and programming help.\n When data has been retrieved, explains findings in accessible terms.') class-attribute instance-attribute

respond(messages, context, step_title=None) async

ChatLogs

Bases: Parameterized

conn = sqlite3.connect(self.filename) instance-attribute

cursor = self.conn.cursor() instance-attribute

filename = param.String(default='chat_logs.db') class-attribute instance-attribute

update_status(message_id, liked=None, disliked=None, removed=None)

upsert(session_id, message_id, message_index, message_user, message_content)

ChatUI

Bases: UI

ChatUI provides a high-level entrypoint to start chatting with your data in a chat based UI.

This high-level wrapper allows providing the data sources you will be chatting with and then configures the assistant and agents.

:Example:

.. code-block:: python

import lumen.ai as lmai

lmai.ChatUI(data='~/data.csv').servable()

llm_choices = param.List(default=[], doc='\n List of available LLM model choices to show in the configuration dialog.') class-attribute instance-attribute

provider_choices = param.Dict(default={}, doc='\n Available LLM providers to show in the configuration dialog.') class-attribute instance-attribute

title = param.String(default='Lumen ChatUI', doc='Title of the app.') class-attribute instance-attribute

Coordinator

Bases: Viewer, VectorLookupToolUser

A Coordinator is responsible for coordinating the actions of a number of agents towards the user defined query by computing an execution graph and then executing each step along the graph.

agents = param.List(default=[ChatAgent], doc='\n List of agents to coordinate.') class-attribute instance-attribute

context = param.Dict(default={}) class-attribute instance-attribute

history = param.Integer(default=3, doc='\n Number of previous user-assistant interactions to include in the chat history.') class-attribute instance-attribute

llm_tools = param.List(default=[make_load_metaset_relevant_docs_tool, make_document_vector_llm_tools, make_monty_llm_tool], doc='\n List of tools for the Planner to make available to the LLM. The tools are also\n made available to the agents.') class-attribute instance-attribute

prompts = param.Dict(default={'main': {}, 'tool_relevance': {'template': PROMPTS_DIR / 'Coordinator' / 'tool_relevance.jinja2', 'response_model': ThinkingYesNo}}) class-attribute instance-attribute

validation_enabled = param.Boolean(default=False, allow_refs=True, doc='\n Whether to enable the ValidationAgent in the planning process.') class-attribute instance-attribute

verbose = param.Boolean(default=False, allow_refs=True, doc='\n Whether to show verbose output.') class-attribute instance-attribute

within_ui = param.Boolean(default=False, constant=True, doc='\n Whether this coordinator is being used within the UI.') class-attribute instance-attribute

respond(messages, context, **kwargs) async

sync(context) async

DeckGLAgent

Bases: BaseCodeAgent

Agent for generating DeckGL 3D map visualizations.

Supports two generation modes: - When code_execution is 'disabled': Generate DeckGL JSON specs directly - When code_execution is enabled: Generate PyDeck Python code, execute safely, convert to spec

conditions = param.List(default=['Use for 3D geographic visualizations, map-based data, or when user requests DeckGL/deck.gl', 'Use for large-scale geospatial data with latitude/longitude coordinates', 'Use for hexbin aggregations, heatmaps, or 3D extruded visualizations on maps', "Can render a geometry column (GeoDataFrame polygons/lines/points) on a basemap, suited to 3D/extruded maps or geometry whose schema is 'geographic' (a lat/lon CRS) so the basemap gives real-world context", 'NOT when the table only names places (a state or country column) without geometry or lat/lon, since deck.gl has nothing to draw and cannot fetch boundaries']) class-attribute instance-attribute

default_map_style = 'https://basemaps.cartocdn.com/gl/dark-matter-gl-style/style.json' class-attribute instance-attribute

prompts = param.Dict(default={'main': {'response_model': DeckGLSpec, 'template': PROMPTS_DIR / 'DeckGLAgent' / 'main.jinja2'}, 'main_pydeck': {'response_model': PyDeckSpec, 'template': PROMPTS_DIR / 'DeckGLAgent' / 'main_pydeck.jinja2'}, 'code_safety': {'response_model': CodeSafetyCheck, 'template': PROMPTS_DIR / 'DeckGLAgent' / 'code_safety.jinja2'}, 'revise_output': {'response_model': RetrySpec, 'template': PROMPTS_DIR / 'DeckGLAgent' / 'revise_output.jinja2'}}) class-attribute instance-attribute

purpose = param.String(default='Generates DeckGL 3D map visualizations from geographic data.') class-attribute instance-attribute

user = param.String(default='DeckGL') class-attribute instance-attribute

view_type = DeckGLView class-attribute instance-attribute

respond(messages, context, step_title=None) async

Generate a DeckGL visualization.

DocumentListAgent

Bases: BaseListAgent

The DocumentListAgent lists all available documents provided by the user.

conditions = param.List(default=['Use when user asks to list or see all available documents', 'NOT when user asks about specific document content']) class-attribute instance-attribute

input_schema = DocumentListInputs class-attribute instance-attribute

purpose = param.String(default='\n Displays a list of all available documents.') class-attribute instance-attribute

user = param.String(default='Documents') class-attribute instance-attribute

applies(context) async classmethod

DocumentSummarizerAgent

Bases: BaseLumenAgent

Synthesizes relevant document context into a user-focused summary.

conditions = param.List(default=['Use when user asks questions answerable from documentation or uploaded metadata files', 'Use for summarization of documentation', 'NOT when user only asks to list available documents']) class-attribute instance-attribute

input_schema = DocumentSummarizerInputs class-attribute instance-attribute

not_with = param.List(default=['DbtslAgent', 'SQLAgent']) class-attribute instance-attribute

output_schema = DocumentSummarizerOutputs class-attribute instance-attribute

prompts = param.Dict(default={'main': {'template': PROMPTS_DIR / 'DocumentSummarizerAgent' / 'main.jinja2'}}) class-attribute instance-attribute

purpose = param.String(default='\n Synthesizes relevant documents into an answer focused on the user query.\n Also renders a document editor with tabbed previews of matched files.') class-attribute instance-attribute

user = param.String(default='Summarizer') class-attribute instance-attribute

applies(context) async classmethod

respond(messages, context, step_title=None) async

DownloadSourceControls

Bases: FileSourceControls

Controls for downloading files from URLs and extracting tabular data.

Supports CSV, JSON, Parquet, Excel, and HTML files. For HTML pages, all tables are extracted and loaded as separate database tables.

Kaggle dataset URLs (e.g. https://www.kaggle.com/datasets/owner/name) are also supported and will download all data files in the dataset into a single source with one table per file.

download_url = param.String(default='', label='Download URL(s)', doc='Enter one or more URLs, one per line, and press <Shift+Enter> to download') class-attribute instance-attribute

input_placeholder = param.String(default='Enter URLs, one per line, and press <Shift+Enter> to download') class-attribute instance-attribute

label = '<span class="material-icons" style="vertical-align: middle;">download</span> Fetch Remote Data' class-attribute instance-attribute

load_mode = 'manual' class-attribute instance-attribute

source_name_prefix = 'DownloadedSource' class-attribute instance-attribute

as_tools(query=None, top_k=5)

Return a tool that allows the agent to download data from URLs.

The tool accepts a URL string and downloads the file, extracting tabular data from CSV, JSON, Parquet, Excel, or HTML files.

DuckDBSource

Bases: BaseSQLSource

DuckDBSource provides a simple wrapper around the DuckDB SQL connector.

To specify tables to be queried provide a list or dictionary of tables. A SQL expression to fetch the data from the table will then be generated using the sql_expr, e.g. if we specify a local table flights.db the default sql_expr SELECT * FROM {table} will expand that to SELECT * FROM flights.db. If you want to specify a full SQL expression as a table you must change the sql_expr to '{table}' ensuring no further templating is applied.

Note that certain functionality in DuckDB requires modules to be loaded before making a query. These can be specified using the initializers parameter, providing the ability to define DuckDb statements to be run when initializing the connection.

connection property

dialect = 'duckdb' class-attribute instance-attribute

ephemeral = param.Boolean(default=False, doc='\n Whether the data is ephemeral, i.e. manually inserted into the\n DuckDB table or derived from real data.') class-attribute instance-attribute

filter_in_sql = param.Boolean(default=True, doc='\n Whether to apply filters in SQL or in-memory.') class-attribute instance-attribute

geometry_crs = param.String(default=None, allow_None=True, doc='\n CRS to reapply to geometry columns after the WKB roundtrip through\n DuckDB, used as a fallback when the column type carries no CRS of its\n own. May be set explicitly for a known dataset; an ingested frame\n instead records its CRS on the GEOMETRY column type itself.') class-attribute instance-attribute

initializers = param.List(default=[], doc='\n SQL statements to run to initialize the connection.') class-attribute instance-attribute

load_schema = param.Boolean(default=True, doc='Whether to load the schema') class-attribute instance-attribute

mirrors = param.Dict(default={}, doc='\n Mirrors the tables into the DuckDB database. The mirrors\n should define a mapping from the table names to the source of\n the mirror which may be defined as a Pipeline or a tuple of\n the source and the table.') class-attribute instance-attribute

read_only = param.Boolean(default=None, doc='\n Whether to open the DuckDB database in read-only mode.') class-attribute instance-attribute

source_type = 'duckdb' class-attribute instance-attribute

sql_expr = param.String(default='SELECT * FROM {table}', doc='\n The SQL expression to execute.') class-attribute instance-attribute

table_params = param.Dict(default={}, doc='\n Dictionary mapping table names to lists of SQL parameters.\n Parameters are used with placeholders (?) in SQL expressions.') class-attribute instance-attribute

tables = param.ClassSelector(class_=(list, dict), doc='\n List or dictionary of tables.') class-attribute instance-attribute

uri = param.String(doc='The URI of the DuckDB database') class-attribute instance-attribute

close()

Close the DuckDB connection, releasing associated resources.

This method should be called when the source is no longer needed to prevent connection leaks and properly clean up server-side resources.

create_sql_expr_source(tables, materialize=True, params=None, **kwargs)

Creates a new SQL Source given a set of table names and corresponding SQL expressions.

Arguments

tables: dict[str, str] Mapping from table name to SQL expression. materialize: bool Whether to materialize new tables params: dict[str, list | dict] | None Optional mapping from table name to parameters: - list: Positional parameters for ? placeholders - dict: Named parameters for $param_name placeholders kwargs: any Additional keyword arguments.

Returns:

Name Type Description
source DuckDBSource

execute(sql_query, params=None, *args, **kwargs)

execute_with_timeout(sql_query, timeout, fetch=False) async

fetch(sql_query, params=None)

Fetch query results in dataframe_backend, built by DuckDB itself.

Not the inherited fetch, which reads through execute. Two reasons: the data path asks for date_as_object so a DATE stays a date rather than becoming a timestamp, and DuckDB can materialise polars and arrow directly. Building the frame in the asked-for library rather than converting a pandas one is most of the point of the parameter: on a 5M row table fetching arrow rather than pandas is an order of magnitude quicker and allocates less than half the memory.

from_df(tables, **kwargs) classmethod

Creates an ephemeral, in-memory DuckDBSource containing the supplied dataframe.

Arguments

tables: dict[str, pandas.DataFrame] A dictionary mapping from table names to DataFrames kwargs: any Additional keyword arguments for the source

Returns:

Name Type Description
source DuckDBSource

from_spec(spec) classmethod

get(table, **query)

get_tables()

missing_table(error)

normalize_table(table)

to_spec(context=None)

ErrorDescription

Bases: BaseModel

Represents a user-facing error explanation.

Style rules: - Do not use first-person pronouns (“I”, “we”). - All outputs must be written in neutral, system-level language.

Guidance: - If you think it's an internal error, indicate that the user should rerun. - Otherwise, ask the user to either replan OR clarify their questions. - You are given the error type, message, the user query, and the plan the planning agent came up with.

explanation = Field(description='A brief description of the error suitable for a non-technical user.', max_length=300) class-attribute instance-attribute

Exploration

Bases: Parameterized

context = param.Dict(allow_refs=True) class-attribute instance-attribute

conversation = Children() class-attribute instance-attribute

initialized = param.Boolean(default=False) class-attribute instance-attribute

parent = param.Parameter() class-attribute instance-attribute

plan = param.ClassSelector(class_=Plan) class-attribute instance-attribute

subtitle = param.String(default='') class-attribute instance-attribute

title = param.String(default='') class-attribute instance-attribute

view = Child() class-attribute instance-attribute

ExplorerUI

Bases: UI

ExplorerUI provides a high-level entrypoint to start chatting with your data in split UI allowing users to load tables, explore them using Graphic Walker, and then interrogate the data via a chat interface.

This high-level wrapper allows providing the data sources you will be chatting with and then configures the assistant and agents.

:Example:

.. code-block:: python

import lumen.ai as lmai

lmai.ExplorerUI(data='~/data.csv').servable()

code_execution = param.Selector(default='hidden', objects=['hidden', 'disabled', 'prompt', 'llm', 'allow'], doc="\n Code execution mode for generating Vega-Lite specs via Altair code.\n Controls whether the code execution selector appears in the UI preferences:\n - hidden: Do not show code execution option in preferences (default)\n - disabled: Show selector, but default to no code execution (Vega-Lite spec only)\n - prompt: Show selector, default to prompting user for permission to execute\n - llm: Show selector, default to LLM-validated code execution\n - allow: Show selector, default to executing code without confirmation\n\n WARNING: The 'prompt', 'llm', and 'allow' modes execute LLM-generated code and\n must NEVER be enabled in production environments with access to secrets, credentials,\n or sensitive data. The LLM validation layer reduces accidental errors but does NOT\n provide meaningful security against malicious or compromised inputs.\n ") class-attribute instance-attribute

title = param.String(default='Lumen AI - Data Explorer', doc='Title of the app.') class-attribute instance-attribute

FileSourceControls

Bases: BaseSourceControls

Intermediate base class for controls that process files (upload/download).

Adds file-card machinery on top of BaseSourceControls lifecycle management. UploadSourceControls and DownloadSourceControls inherit from this class.

add = param.Event(doc='Use uploaded file(s)') class-attribute instance-attribute

add_button_icon = param.String(default='add', doc='\n Material icon name for the add/upload confirmation button.') class-attribute instance-attribute

add_button_label = param.String(default='Confirm file(s)', doc='\n Text label for the add/upload confirmation button.') class-attribute instance-attribute

clear_uploads = param.Boolean(default=True, doc='Clear uploaded file tabs') class-attribute instance-attribute

filedropper_kwargs = param.Dict(default={}, doc="Keyword arguments to pass to FileDropper.\n Common options include 'accepted_filetypes' and 'max_file_size'.\n See https://panel.holoviz.org/reference/widgets/FileDropper.html for all options.") class-attribute instance-attribute

multiple = param.Boolean(default=True, doc='Allow multiple files') class-attribute instance-attribute

replace_controls = param.Boolean(default=False, doc='Replace controls on add') class-attribute instance-attribute

upload_handlers = param.Dict(default={}, doc='Handlers for custom file extensions') class-attribute instance-attribute

LLMConfigDialog

Bases: Viewer

Dialog component for LLM configuration.

Provides: - Provider selection - Temperature slider - Model selection for different model types - Apply/Cancel buttons

agent_types = param.List(default=[], doc='Available agents in the UI') class-attribute instance-attribute

llm = param.ClassSelector(class_=Llm, doc='The LLM instance to configure') class-attribute instance-attribute

llm_choices = param.List(default=[], doc='Available model choices to display in dropdowns') class-attribute instance-attribute

on_llm_change = param.Callable(default=None, doc='Callback function called when LLM provider changes') class-attribute instance-attribute

open property writable

Get dialog open state.

provider_choices = param.Dict(default={}, doc='Available LLM providers to choose from') class-attribute instance-attribute

close()

Close the dialog.

Llm

Bases: Parameterized

Base class for LLM implementations with standardized client caching.

Subclasses MUST implement _create_base_client(). Subclasses MAY set _instructor_wrapper (default: "openai"). Subclasses MAY override _create_instructor_client() or _get_completion_method().

create_kwargs = param.Dict(default={'max_retries': 1}, doc='\n Additional keyword arguments to pass to the LLM provider\n when calling chat.completions.create. Defaults to no instructor retries\n since agents handle retries themselves.') class-attribute instance-attribute

display_name = param.String(default='', constant=True, doc='Display name for UI') class-attribute instance-attribute

interceptor = param.ClassSelector(default=None, class_=Interceptor, doc='\n Intercepter instance to intercept LLM calls, e.g. for logging.') class-attribute instance-attribute

logfire_tags = param.List(default=None, doc='\n Whether to log LLM calls and responses to logfire.\n If a list of tags is provided, those tags will be used for logging.\n Suppresses streaming responses if enabled since\n logfire does not track token usage on stream.') class-attribute instance-attribute

mode = param.Selector(default=(Mode.JSON_SCHEMA), objects=BASE_MODES, doc='\n The calling mode used by instructor to guide the LLM towards generating\n outputs matching the schema.') class-attribute instance-attribute

model_kwargs = param.Dict(default={}, doc='\n LLM model definitions indexed by type. Supported types include\n \'default\', \'reasoning\' and \'sql\'. Agents may pick which model to\n invoke for different reasons.\n\n An entry may declare an optional \'description\' (shown to the routing\n model so it can reason about what each option is for) and an optional\n \'routing\' key to opt in to model routing for that type: the routing\n model is invoked first to pick which entry to actually use for each\n call, e.g.\n {"default": {"model": "gpt-5.4-mini"},\n "edit": {"model": "gpt-5.2",\n "description": "Best for editing tables and visualizations",\n "routing": {"model": "nemotron-switchyard"}}}') class-attribute instance-attribute

select_models = param.List(default=[], constant=True, doc='Available models for selection dropdowns') class-attribute instance-attribute

spec_descriptions = param.Dict(default=SPEC_DESCRIPTIONS, doc='\n Mapping of spec key to human-readable description, used as a\n fallback in the routing prompt when ``model_kwargs`` entries\n do not declare their own ``description``. Seeded with the spec\n keys every provider shares and enriched by the UI layer from\n agent class metadata.') class-attribute instance-attribute

temperature = param.Number(default=0.7, bounds=(0, None), allow_None=True, constant=True, doc='\n The temperature to use for sampling. Set to None to omit the\n temperature from requests entirely, e.g. for models that reject it.') class-attribute instance-attribute

timeout = param.Number(default=120, bounds=(1, None), constant=True, doc='\n The timeout in seconds for API calls.') class-attribute instance-attribute

tools = param.List(default=[], doc='\n Default tools that are always available to this LLM instance.\n These are combined with any tools passed directly to invoke()\n or stream() calls. Accepts the same types as the tools argument\n on those methods (dicts, FunctionTool, or MCPTool instances).') class-attribute instance-attribute

usage = UsageCollector() instance-attribute

usage_pricing = param.Dict(default={}, doc='USD per million tokens by model:\n {"model-name": {"input": 1.0, "cached": 0.1, "cache_write": 1.25,\n "output": 4.0}}.\n Unknown models retain token counts but have no estimated cost.') class-attribute instance-attribute

capture_usage()

Capture provider usage for this LLM in the current async context.

initialize(log_level) async

invoke(messages, system='', response_model=None, allow_partial=False, model_spec='default', tools=None, submit_tool=None, **input_kwargs) async

Invokes the LLM and returns its response.

Arguments

messages: list[Message] A list of messages to feed to the LLM. system: str A system message to provide to the LLM. response_model: BaseModel | None A Pydantic model that the LLM should materialize. allow_partial: bool Whether to allow the LLM to only partially fill the provided response_model. tools: list[dict[str, Any] | FunctionTool | MCPTool] | None Tool definitions, FunctionTool, or MCPTool instances to pass through. When response_model is also given, the client runs tool calls in a loop until none are requested, then requests the structured response (tools may be unused). Tool errors, unknown tools and invalid arguments are returned to the model as tool results; max_tool_rounds (default 16) caps the rounds. submit_tool: SubmitTool | None Offer response_model as a tool so the model can answer inside the tool loop. Requires response_model. model: Literal['default' | 'reasoning' | 'sql'] The model as listed in the model_kwargs parameter to invoke to answer the query.

Returns:

Type Description
The completed response_model.

models()

Return the set of available model identifiers from this provider.

By default returns an empty set. Subclasses should override to query their provider's API for available models.

Returns:

Type Description
set[str]

Set of available model identifiers.

run_client(model_spec, messages, **kwargs) async

stream(messages, system='', response_model=None, field=None, model_spec='default', tools=None, **kwargs) async

Invokes the LLM and streams its response.

Arguments

messages: list[Message] A list of messages to feed to the LLM. system: str A system message to provide to the LLM. response_model: type[BaseModel] | None A Pydantic model that the LLM should materialize. field: str The field in the response_model to stream. tools: list[dict[str, Any] | FunctionTool | MCPTool] | None Tool definitions, FunctionTool, or MCPTool instances to pass through. model: Literal['default' | 'reasoning' | 'sql'] The model as listed in the model_kwargs parameter to invoke to answer the query.

Yields:

Type Description
The string or response_model field.

trace()

Capture model round trips and tool calls in the current async context.

warmup(model_kwargs) classmethod

Allows LLM provider to perform actions that ensure that the model(s) are ready to run, e.g. downloading the model files.

LumenEditor

Bases: Viewer

component = param.ClassSelector(class_=Component) class-attribute instance-attribute

editor = self._render_editor() instance-attribute

export_formats = ('yaml',) class-attribute instance-attribute

footer = param.List(default=[]) class-attribute instance-attribute

language = 'yaml' class-attribute instance-attribute

loading = param.Boolean() class-attribute instance-attribute

spec = param.String(allow_None=True) class-attribute instance-attribute

title = param.String(allow_None=True) class-attribute instance-attribute

view = ParamMethod(self.render, inplace=True, sizing_mode='stretch_width') instance-attribute

export(fmt)

render() async

render_context() async

render_controls(task, interface)

validate_spec(spec) classmethod

OpenAI

Bases: Llm, OpenAIMixin

An LLM implementation using the OpenAI cloud.

api = param.Selector(default='chat_completions', objects=['chat_completions', 'responses'], doc='\n OpenAI API primitive to use.\n - ``chat_completions``: Uses ``/v1/chat/completions`` (default)\n - ``responses``: Uses ``/v1/responses``') class-attribute instance-attribute

display_name = param.String(default='OpenAI', constant=True) class-attribute instance-attribute

mode = param.Selector(default=(Mode.TOOLS)) class-attribute instance-attribute

model_kwargs = param.Dict(default={'default': {'model': 'gpt-6-luna'}}) class-attribute instance-attribute

select_models = param.List(default=['gpt-6-luna', 'gpt-5.6-luna', 'gpt-6-sol', 'gpt-5.4-nano'], constant=True, doc='Warning: Reasoning models (gpt-5, o4-mini) are much slower and not suitable for dialog interfaces.') class-attribute instance-attribute

temperature = param.Number(default=0.25, bounds=(0, None), allow_None=True, constant=True) class-attribute instance-attribute

get_client(model_spec, response_model=None, **kwargs) async

models()

Return the set of available model identifiers from OpenAI.

run_client(model_spec, messages, **kwargs) async

Pipeline

Bases: Viewer, Component

Pipeline encapsulates filters and transformations applied to a :class:lumen.sources.base.Source table.

A Pipeline ingests data from a :class:lumen.sources.base.Source table or another Pipeline applying the declared :class:lumen.filters.base.Filter, :class:lumen.transforms.base.Transform and :class:lumen.transforms.sql.SQLTransform definitions. It can be used to drive one or more visual outputs or leveraged as a standalone component to encapsulate multiple data processing steps.

auto_update = param.Boolean(default=True, constant=True, doc='\n Whether changes in filters, transforms and references automatically\n trigger updates in the data or whether an update has to be triggered\n manually using the update event or the update button in the UI.') class-attribute instance-attribute

control_panel property

data = DataFrame(doc='The current data on this source.') class-attribute instance-attribute

dataframe_backend = param.Selector(default=None, objects=[None, *DATAFRAME_BACKENDS], doc='\n The dataframe library `Pipeline.data` is returned as. Leave unset to\n get whatever the Source produced, which avoids a conversion.') class-attribute instance-attribute

filters = param.List(item_type=Filter, doc='\n A list of Filters to apply to the source data.') class-attribute instance-attribute

pipeline = param.ClassSelector(class_=None, doc='\n Optionally a pipeline may be chained to another pipeline.') class-attribute instance-attribute

refs property

schema = param.Dict(doc='The schema of the input data.') class-attribute instance-attribute

source = param.ClassSelector(class_=Source, doc='\n The Source this pipeline is fed by.') class-attribute instance-attribute

sql_pushdown = param.Boolean(default=True, doc='\n Whether to narrow the query to the columns the transforms actually\n need when the Source supports SQL, instead of fetching every column\n and projecting in pandas. Only applied where the result is provably\n identical; set to False to always fetch the full table.') class-attribute instance-attribute

sql_transforms = param.List(item_type=SQLTransform, doc='\n A list of SQLTransforms to apply to the source data.') class-attribute instance-attribute

table = param.String(doc='\n The name of the table driving this pipeline.') class-attribute instance-attribute

transforms = param.List(item_type=Transform, doc='\n A list of Transforms to apply to the source data.') class-attribute instance-attribute

update = param.Event(label='Apply update', doc='\n Update event trigger (if manual update is set).') class-attribute instance-attribute

add_filter(filt, field=None, **kwargs)

Add a filter to the pipeline.

Arguments

filt: Filter | Type[Filter] The filter instance or filter type to add. field: str | None The field to filter on (required to instantiate Filter type).

add_transform(transform, **kwargs)

Add a (SQL)Transform to the pipeline.

Arguments

filt: Transform The Transform instance to add.

chain(filters=None, transforms=None, sql_transforms=None, **kwargs)

Chains additional filtering, transform and sql_transform operations on an existing pipeline. Note that if one or more sql_transforms are provided the existing table will be mirrored into a DuckDB database.

Arguments

filters: List[Filter] | None Additional filters to apply on top of existing pipeline. transforms: List[Transform] | None Additional transforms to apply on top of existing pipeline. sql_transforms: List[SQLTransform] | None Additional filters to apply on top of existing pipeline.

Returns:

Type Description
Pipeline

clone(**params)

Create a new instance of the pipeline with optionally overridden parameter values.

from_spec(spec, source=None, source_filters=None) classmethod

Creates a Pipeline from a specification.

Parameters:

Name Type Description Default
spec dict or str

Specification declared as a dictionary of parameter values or a string referencing a source in the sources dictionary.

required

Returns:

Type Description
Resolved and instantiated Pipeline object

get_dataset()

Return the pipeline data as a gridded xarray.Dataset when the source can produce one (xarray-backed, via to_dataset) and this is a plain source query -- no chained pipeline, ParamFilter, or Python transforms. Returns None otherwise so callers fall back to the long-form frame.

Lets gridded views render from a compact xarray grid instead of pivoting the long-form data back to xarray.

get_schema()

Generates a JSON schema for the current data held by the Pipeline.

Returns:

Name Type Description
schema dict[str, any]

JSON schema for each column in the current data.

precache(queries)

Populates the cache of the :class:lumen.sources.base.Source with the provided queries.

Queries can be provided in two formats:

  • A dictionary containing 'filters' and 'variables' dictionaries each containing lists of values to compute a cross-product for, e.g.

    { 'filters': { ': ['a', 'b', 'c', ...], ... }, 'variables': { : [0, 2, 4, ...], ... } } - A list containing dictionaries of explicit values for each filter and variables.

    [{ 'filters': {: 'a'}, 'variables': {: 0} }, { 'filters': {: 'a'}, 'variables': {: 1} }, ... ]

to_spec(context=None)

Exports the full specification to reconstruct this component.

Parameters:

Name Type Description Default
context dict[str, Any] | None

Context contains the specification of all previously serialized components, e.g. to allow resolving of references.

None

Returns:

Type Description
Declarative specification of this component.

traverse(type)

Returns all Filter or Transform objects in a potentially chained pipeline.

validate(spec, context=None) classmethod

Plan

Bases: Section

A Plan is a Task that is a collection of other Tasks.

abort_on_error = param.Boolean(default=True, doc='\n If True, the report will abort if an error occurs.') class-attribute instance-attribute

agents = param.List(item_type=Actor, default=[]) class-attribute instance-attribute

coordinator = param.ClassSelector(class_=(param.Parameterized)) class-attribute instance-attribute

interface = param.ClassSelector(class_=ChatFeed) class-attribute instance-attribute

is_followup = param.Boolean(default=False) class-attribute instance-attribute

execute(context=None, **kwargs) async

render_task_history(i=None, failed=False)

Planner

Bases: Coordinator

The Planner develops a plan to solve the user query step-by-step and then executes it.

planner_tools = param.List(default=[MetadataLookup], doc='\n List of tools to use to provide context for the planner prior\n to making a plan.') class-attribute instance-attribute

prompts = param.Dict(default={'main': {'template': PROMPTS_DIR / 'Planner' / 'main.jinja2', 'response_model': make_plan_model}, 'follow_up': {'template': PROMPTS_DIR / 'Planner' / 'follow_up.jinja2', 'response_model': FollowUpClassification}, 'clarification_check': {'template': PROMPTS_DIR / 'Planner' / 'clarification_check.jinja2', 'response_model': ThinkingYesNo}}) class-attribute instance-attribute

sync(context) async

Sync both main tools and planner tools.

SQLAgent

Bases: BaseLumenAgent

clean_data = param.Boolean(default=True, doc='\n Profile a bounded sample of each query result and, when it contains data-quality problems,\n spend one extra LLM call rewriting the query to clean them up.\n Profiling is deterministic and reuses the frame validation already\n fetched, so a clean result costs nothing; set to False to always take\n the query exactly as first written.') class-attribute instance-attribute

conditions = param.List(default=['Use for querying, filtering, aggregating, or transforming data with SQL', "Use for calculations that require executing SQL (e.g., 'calculate average', 'sum by category')", "Use when user asks to 'show', 'get', 'fetch', 'query', 'find', 'filter', 'calculate', 'aggregate', or 'transform' data", 'Use after external data has been fetched and the user expects the fetched fields/rows to be presented, selected, filtered, or otherwise queried', 'Use when user asks about data quality, completeness, missing values, duplicates, or outliers', "NOT when user asks to 'explain', 'interpret', 'analyze', 'summarize', or 'comment on' existing data", 'NOT useful if the user is using the same data for plotting']) class-attribute instance-attribute

exclusions = param.List(default=['dbtsl_metaset']) class-attribute instance-attribute

input_schema = SQLInputs class-attribute instance-attribute

llm_tools = param.List(default=[make_apply_filter_llm_tool]) class-attribute instance-attribute

max_request_llm_calls = param.Integer(default=40, bounds=(1, None), doc='\n Maximum LLM calls one SQL request may spend across all attempts, tool\n rounds and revisions. Checked before each attempt.') class-attribute instance-attribute

not_with = param.List(default=['DbtslAgent', 'MetadataLookup', 'TableListAgent']) class-attribute instance-attribute

output_schema = SQLOutputs class-attribute instance-attribute

prompts = param.Dict(default={'main': {'response_model': make_sql_model, 'template': PROMPTS_DIR / 'SQLAgent' / 'main.jinja2'}, 'clean_data': {'response_model': SQLCleanup, 'template': PROMPTS_DIR / 'SQLAgent' / 'clean_data.jinja2'}, 'revise_output': {'response_model': RetrySpec, 'template': PROMPTS_DIR / 'SQLAgent' / 'revise_output.jinja2'}}) class-attribute instance-attribute

purpose = param.String(default='\n Creates and executes SQL queries to retrieve, filter, aggregate, or transform data.\n Handles table joins, WHERE clauses, GROUP BY, calculations, and other SQL operations.\n Generates new data pipelines from SQL transformations.\n Profiles every result for data-quality problems (missing values, duplicates,\n placeholder numbers, outliers) and reports them.') class-attribute instance-attribute

query_timeout = param.Number(default=60, bounds=(0, None), allow_None=True, doc="\n Seconds a query run on the model's behalf may take before it is\n abandoned, or None to wait indefinitely. A timed out query is\n revised once; a second timeout ends the request.") class-attribute instance-attribute

request_timeout = param.Number(default=300, bounds=(0, None), allow_None=True, doc='\n Wall-clock limit in seconds for one SQL request across all attempts.\n None disables the limit.') class-attribute instance-attribute

schema_max_tokens = param.Integer(default=4000, allow_None=True, doc='\n Token budget for the data summary in the prompt. Tables beyond it are\n listed with names and types only, and the model loads the rest with\n load_table_schemas.') class-attribute instance-attribute

schema_tables_shown = param.Integer(default=25, bounds=(1, None), doc='\n Most relevant tables described in the data summary; the rest are\n listed by name. schema_max_tokens usually binds first.') class-attribute instance-attribute

stats_wait = param.Number(default=10, allow_None=True, doc='\n Seconds to wait for column statistics that are still being computed\n before answering with names and types; profiling continues in the\n background for later questions. None waits until they are ready.') class-attribute instance-attribute

user = param.String(default='SQL') class-attribute instance-attribute

respond(messages, context, step_title=None) async

Generate SQL in one step (catalog browse, schema load, optional exploration, then structured query).

revise(instruction, messages, context, view=None, spec=None, language=None, errors=None, max_retries=2, **kwargs) async

SQLEditor

Bases: LumenEditor

export_formats = ('sql', 'csv', 'xlsx', 'json', 'markdown') class-attribute instance-attribute

language = 'sql' class-attribute instance-attribute

export(fmt)

render_context() async

render_controls(task, interface)

render_explorer()

Section

Bases: TaskGroup

A Section is a TaskGroup representing a sequence of related tasks.

level = 2 class-attribute instance-attribute

SourceAgent

Bases: Agent

Fetches data from external APIs, Python callables, or URL templates configured via source controls.

The agent reads available actions from SourceLookup context, wraps the relevant callables as FunctionTool instances, and passes them to the LLM. The LLM decides which action to call and fills in parameter values. Results are normalized to DataFrames and registered as DuckDBSource entries for downstream agents (e.g. SQLAgent, hvPlotAgent) to consume.

conditions = param.List(default=['Use when no existing data source can answer the query and external source controls are configured', 'Use when the user asks for data from an external API or service', 'Use when the user provides a URL to read, fetch, discuss, or extract content from — the content must be fetched before any other agent can address it', "Use when the loaded data does not cover the requested scope (e.g. user asks for 'whole month' but data summary only shows a week)", 'Prefer over metadata discovery when no data sources are loaded but source controls exist', 'This agent fetches and registers data or indexes page content as documents — always follow with a presentation, query, or chat step', 'For HTML pages, this agent extracts both tabular data and page text (indexed as a searchable document for ChatAgent)', 'Output is queryable tabular data and/or indexed documents — enables filtering, aggregation, joins, transformation, and conversation by downstream agents', "NOT for listing configured external source actions or answering 'what data is available' before a concrete dataset request", 'NOT when the loaded data already covers the requested scope and user just wants to query/visualize it']) class-attribute instance-attribute

input_schema = SourceInputs class-attribute instance-attribute

output_schema = SourceOutputs class-attribute instance-attribute

prompts = param.Dict(default={'main': {'template': PROMPTS_DIR / 'SourceAgent' / 'main.jinja2'}}) class-attribute instance-attribute

purpose = param.String(default='\n Fetches data from configured external sources (APIs, Python callables,\n URL templates). Discovers relevant endpoints, fills parameters via\n LLM, executes the call, and registers the resulting DataFrame for\n downstream querying and visualization. For HTML pages, also indexes\n page text as a searchable document for conversational agents.') class-attribute instance-attribute

user = param.String(default='Source') class-attribute instance-attribute

applies(context) async classmethod

respond(messages, context, step_title=None) async

SourceCatalog

Bases: Viewer

A component that displays all data sources and documents as hierarchical trees.

context = param.Dict(default={}) class-attribute instance-attribute

sources = param.List(default=[], doc='\n List of data sources to display in the catalog.') class-attribute instance-attribute

vector_store = param.Parameter(default=None, doc='\n The vector store to sync metadata documents to. If not provided,\n metadata files will only be stored locally.') class-attribute instance-attribute

visibility_changed = param.Event(doc='\n Triggered when table or document visibility changes.') class-attribute instance-attribute

sync(context=None)

StoryReport

Bases: Report

A Report with an extra Story tab: an AI writes a blog-post story that interleaves prose with the report's charts and tables, which can then be edited by hand or with the LLM, regenerated, and kept as versions.

reset(start=0)

TableExplorer

Bases: Viewer

TableExplorer provides a high-level entrypoint to explore tables in a split UI. It allows users to load tables, explore them using Graphic Walker, and then interrogate the data via a chat interface.

add_exploration = param.Event(label='Explore table') class-attribute instance-attribute

context = param.Dict(default={}) class-attribute instance-attribute

source_map = {} instance-attribute

table_slug = param.Selector(label='Select table(s) to preview.') class-attribute instance-attribute

create_sql_output()

sync(context=None) async

TableListAgent

Bases: BaseListAgent

The TableListAgent lists all available data and lets the user pick one.

conditions = param.List(default=["Use when user explicitly asks to 'list data', 'show available data', or 'what data do you have'", 'NOT for showing actual data contents, querying, or analyzing data', 'NOT for describing data sources or telling what the data is about']) class-attribute instance-attribute

input_schema = TableListInputs class-attribute instance-attribute

not_with = param.List(default=['DbtslAgent', 'SQLAgent']) class-attribute instance-attribute

purpose = param.String(default='\n Displays a list of all available data & datasets. Not useful for identifying which dataset to use for analysis.') class-attribute instance-attribute

user = param.String(default='Tables') class-attribute instance-attribute

applies(context) async classmethod

UI

Bases: Viewer

UI provides a baseclass and high-level entrypoint to start chatting with your data.

agents = param.List(default=[], doc='\n List of additional Agents to add beyond the default_agents.') class-attribute instance-attribute

analyses = param.List(default=[], doc='\n List of custom analyses. If provided the AnalysesAgent will be added.') class-attribute instance-attribute

context = param.Dict(default={}) class-attribute instance-attribute

coordinator = param.ClassSelector(class_=Coordinator, default=Planner, is_instance=False, doc='\n The Coordinator class that will be responsible for coordinating the Agents.') class-attribute instance-attribute

coordinator_params = param.Dict(default={}, doc='Keyword arguments to pass to the Coordinator.') class-attribute instance-attribute

default_agents = param.List(default=[TableListAgent, ChatAgent, DocumentListAgent, DocumentSummarizerAgent, SQLAgent, SourceAgent, VegaLiteAgent, ValidationAgent, DeckGLAgent], doc='List of default agents which will always be added.') class-attribute instance-attribute

demo_inputs = param.List(default=DEMO_MESSAGES, doc='\n List of instructions to demo the Coordinator.') class-attribute instance-attribute

document_vector_store = param.ClassSelector(class_=VectorStore, default=None, doc='\n The vector store to use for document-related tools. If not provided, a new one will be created\n or inferred from the tools provided.') class-attribute instance-attribute

export_functions = param.Dict(default={}, doc='\n Dictionary mapping from name of exporter to export function.') class-attribute instance-attribute

filedropper_kwargs = param.Dict(default={}, doc="Keyword arguments to pass to FileDropper in UploadSourceControls.\n Common options include 'accepted_filetypes' and 'max_file_size'.\n See https://panel.holoviz.org/reference/widgets/FileDropper.html for all available options.") class-attribute instance-attribute

interface = param.ClassSelector(class_=ChatFeed, doc='\n The interface for the Coordinator to interact with.') class-attribute instance-attribute

llm = param.ClassSelector(class_=Llm, default=None, doc='\n The LLM provider to use. Detected from the environment when the UI\n is created if not provided.') class-attribute instance-attribute

llm_choices = param.List(default=[], doc='\n List of available LLM model choices to show in the configuration dialog.') class-attribute instance-attribute

llm_tools = param.List(default=[], doc="\n Extra tools for the coordinator's :attr:`~lumen.ai.actor.LLMUser.llm_tools`\n (expanded in ``_invoke_prompt`` / ``_stream_prompt`` only).") class-attribute instance-attribute

log_level = param.ObjectSelector(default='DEBUG', objects=['DEBUG', 'INFO', 'WARNING', 'ERROR'], doc='\n The log level to use.') class-attribute instance-attribute

logfire_tags = param.List(default=None, doc='\n Whether to log LLM calls and responses to logfire.\n If a list of tags is provided, those tags will be used for logging.\n Suppresses streaming responses if enabled since\n logfire does not track token usage on stream.') class-attribute instance-attribute

logs_db_path = param.String(default=None, constant=True, doc='\n The path to the log file that will store the messages exchanged with the LLM.') class-attribute instance-attribute

notebook_preamble = param.String(default='', doc='\n Preamble to add to exported notebook(s).') class-attribute instance-attribute

page_config = param.Dict(default={}, doc='\n Configuration for the panel-material-ui Page component the UI is rendered into.') class-attribute instance-attribute

provider_choices = param.Dict(default={}, doc='\n Available LLM providers to show in the configuration dialog.') class-attribute instance-attribute

source_controls = param.List(default=[UploadSourceControls, DownloadSourceControls], doc='\n List of SourceControls types to manage datasets.') class-attribute instance-attribute

suggestions = param.List(default=GETTING_STARTED_SUGGESTIONS, doc='\n Initial list of suggestions of actions the user can take.') class-attribute instance-attribute

title = param.String(default='Lumen UI', doc='Title of the app.') class-attribute instance-attribute

tools = param.List(doc='\n List of Tools that can be invoked by the coordinator.') class-attribute instance-attribute

upload_handlers = param.Dict(default={}, doc='\n Dictionary mapping from file extensions to handler function,\n e.g. {"hdf5": ...}. The callback function should accept the file bytes,\n table alias, and filename, add or modify the `sources` in memory, and return a bool\n (True if the table was successfully uploaded).') class-attribute instance-attribute

vector_store = param.ClassSelector(class_=VectorStore, default=None, doc='\n The vector store to use for tools unrelated to documents. If not provided, a new one will be created\n or inferred from the tools provided.') class-attribute instance-attribute

servable(title=None, **kwargs)

show(**kwargs)

UploadSourceControls

Bases: FileSourceControls

Controls for uploading files from the local filesystem.

add_button_icon = 'upload_file' class-attribute instance-attribute

add_button_label = 'Upload file(s)' class-attribute instance-attribute

label = '<span class="material-icons" style="vertical-align: middle;">upload</span> Upload Data' class-attribute instance-attribute

load_mode = 'manual' class-attribute instance-attribute

ValidationAgent

Bases: Agent

ValidationAgent focuses solely on validating whether the executed plan fully answered the user's original query. It identifies missing elements and suggests next steps when validation fails.

conditions = param.List(default=['Use to validate whether executed plans fully answered user queries', 'Use to identify missing elements from the original user request', 'NOT for data analysis, pattern identification, or technical programming questions']) class-attribute instance-attribute

input_schema = ValidationInputs class-attribute instance-attribute

output_schema = ValidationOutputs class-attribute instance-attribute

prompts = param.Dict(default={'main': {'template': PROMPTS_DIR / 'ValidationAgent' / 'main.jinja2', 'response_model': QueryCompletionValidation, 'tools': []}}) class-attribute instance-attribute

purpose = param.String(default="\n Validates whether executed plans fully answered the user's original query.\n Identifies missing elements, assesses completeness, and suggests next steps\n when validation fails. Acts as a quality gate for plan execution.") class-attribute instance-attribute

user = param.String(default='Validation') class-attribute instance-attribute

respond(messages, context, step_title=None) async

VectorStore

Bases: LLMUser

Abstract base class for a vector store.

chunk_func = param.Callable(default=None, doc='\n The function used to split documents into chunks.\n Must accept `text`. If None, defaults to semchunk.chunkerify\n with the chunk_size.') class-attribute instance-attribute

chunk_func_kwargs = param.Dict(default={}, doc='\n Additional keyword arguments to pass to the chunking function.') class-attribute instance-attribute

chunk_size = param.Integer(default=512, doc='Maximum size of text chunks to split documents into.') class-attribute instance-attribute

chunk_tokenizer = param.String(default='gpt-4o-mini', doc='\n If using the default chunk_func, the tokenizer used to split documents into chunks.\n Must be a valid tokenizer name from the transformers or tiktoken library.\n Otherwise, please pass in a custom chunk_func.') class-attribute instance-attribute

embeddings = param.ClassSelector(class_=Embeddings, default=(NumpyEmbeddings()), allow_None=True, doc='Embeddings object for text processing.') class-attribute instance-attribute

excluded_metadata = param.List(default=['llm_context'], doc='List of metadata keys to exclude when creating the embeddings.') class-attribute instance-attribute

max_concurrent = param.Integer(default=5, bounds=(1, None), doc="\n Maximum number of files to process concurrently in add_directory.\n Set to 1 for backends that don't support concurrent writes.") class-attribute instance-attribute

prompts = param.Dict(default={'main': {'template': PROMPTS_DIR / 'VectorStore' / 'main.jinja2'}, 'should_situate': {'template': PROMPTS_DIR / 'VectorStore' / 'should_situate.jinja2', 'response_model': YesNo}}, doc="\n A dictionary of prompts used by the vector store, indexed by prompt name.\n Each prompt should be defined as a dictionary containing a template\n 'template' and optionally a 'model' and 'tools'.") class-attribute instance-attribute

situate = param.Boolean(default=False, doc='\n Whether to insert a `llm_context` key in the metadata containing\n contextual about the chunks.') class-attribute instance-attribute

add(items, force_ids=None, situate=None) async

Add items to the vector store.

Parameters:

Name Type Description Default
items list[dict]

List of dictionaries containing 'text' and optional 'metadata'.

required
force_ids list[int] | None

Optional list of IDs to use instead of generating new ones.

None
situate bool | None

Whether to insert a llm_context key in the metadata containing contextual about the chunks. If None, uses the class default.

None

Returns:

Type Description
List of assigned IDs for the added items.

add_directory(directory, pattern='*', exclude_patterns=None, metadata=None, situate=None, upsert=False, raise_on_error=False, max_concurrent=None) async

Recursively add files from a directory that match the pattern and don't match exclude patterns.

Parameters:

Name Type Description Default
directory Union[str, PathLike]

The path to the directory to search for files.

required
pattern str

Glob pattern to match files against (e.g., ".txt", ".py"). Default is "*".

'*'
exclude_patterns Optional[List[str]]

List of patterns to exclude. Files matching any of these patterns will be skipped.

None
metadata Optional[dict[str, Any]]

Base metadata to apply to all files. Will be extended with filename-specific metadata.

None
situate Optional[bool]

Whether to insert a llm_context key in the metadata. If None, uses the class default.

None
upsert bool

If True, will update existing items if similar content is found. Default is False.

False
raise_on_error

If True, will raise an error if any file fails to process. Default is False.

False
max_concurrent int | None

Maximum number of files to process concurrently. If None, uses self.max_concurrent.

None

Returns:

Type Description
List[int]

Combined list of IDs for all added files.

add_file(filename, ext=None, metadata=None, situate=None, upsert=False) async

Adds a file or a URL to the collection.

Parameters:

Name Type Description Default
filename str | IO | Any | PathLike

The path to the file, a file-like object or a URL to be added.

required
ext str | None

The file extension to associate with the added file. If not provided, it will be determined from the file or URL.

None
metadata dict | None

A dictionary containing metadata related to the file (e.g., title, author, description). Defaults to None.

None
situate bool | None

Whether to insert a llm_context key in the metadata containing contextual about the chunks. If None, uses the class default.

None
upsert bool

If True, will update existing items if similar content is found. Defaults to False.

False

Returns:

Type Description
List of assigned IDs for the added items.

clear() abstractmethod

Clear all items from the vector store.

close() abstractmethod

Close the vector store and release any resources.

delete(ids) abstractmethod

Delete items from the vector store by their IDs.

Parameters:

Name Type Description Default
ids list[int]

List of IDs to delete.

required

filter_by(filters, limit=None, offset=0) abstractmethod

Filter items by metadata without using embeddings similarity.

Parameters:

Name Type Description Default
filters dict

Dictionary of metadata key-value pairs to filter by.

required
limit int | None

Maximum number of results to return. If None, returns all matches.

None
offset int

Number of results to skip (for pagination).

0

Returns:

Type Description
List of results with 'id', 'text', and 'metadata'.

query(text, top_k=5, filters=None, threshold=-1.0) abstractmethod async

Query the vector store for similar items.

Parameters:

Name Type Description Default
text str

The query text.

required
top_k int

Number of top results to return.

5
filters dict | None

Optional metadata filters.

None
threshold float

Minimum similarity score required for a result to be included.

-1.0

Returns:

Type Description
List of results with 'id', 'text', 'metadata', and 'similarity' score.

should_situate_chunk(chunk) async

Determine whether a chunk should be situated based on its content.

Parameters:

Name Type Description Default
chunk str

The chunk text to evaluate

required

Returns:

Type Description
bool

Whether the chunk should be situated

upsert(items, situate=None) async

Add items to the vector store if similar items don't exist, update them if they do.

Parameters:

Name Type Description Default
items list[dict]

List of dictionaries containing 'text' and optional 'metadata'.

required
situate bool | None

Whether to insert a llm_context key in the metadata containing contextual about the chunks. If None, uses the class default.

None

Returns:

Type Description
List of assigned IDs for the added or updated items.

VegaLiteAgent

Bases: BaseCodeAgent

conditions = param.List(default=['Use for publication-ready visualizations or when user specifically requests Vega-Lite charts', 'Use for polished charts intended for presentation or sharing', 'Use for a choropleth whose table names places (states, countries) but holds no geometry or coordinates, since the boundaries can be looked up and joined by name']) class-attribute instance-attribute

doc_pages_per_slug = param.Integer(default=2, bounds=(1, None), doc='\n Maximum number of retrieved sections allowed from any single\n documentation page. Without a cap, similarity collapses onto one page:\n a "bar chart" query returns five near-identical mark-bar variants while\n the transform and encoding pages that would broaden the answer sit just\n below the cut.') class-attribute instance-attribute

exclude_spec_constructs = param.List(default=['facet', 'repeat', 'concat', 'hconcat', 'vconcat', 'row', 'column'], item_type=str, doc='\n Vega-Lite composition keys that disqualify a retrieved example spec.\n The declarative prompt tells the model not to pack several plots into one\n entry with these, so injecting a reference spec that uses them puts the\n documentation at odds with the instructions. Set to [] to keep every\n retrieved spec -- appropriate for the Altair path, whose prompt permits\n composition within a single chart entry.') class-attribute instance-attribute

n_doc_pages = param.Integer(default=5, bounds=(0, None), doc='\n Number of documentation sections to inject into the prompt.\n Set to 0 to disable documentation lookup entirely.') class-attribute instance-attribute

prompts = param.Dict(default={'main': {'response_model': VegaLiteSpec, 'template': PROMPTS_DIR / 'VegaLiteAgent' / 'main.jinja2'}, 'main_altair': {'response_model': AltairSpec, 'template': PROMPTS_DIR / 'VegaLiteAgent' / 'main_altair.jinja2'}, 'code_safety': {'response_model': CodeSafetyCheck, 'template': PROMPTS_DIR / 'VegaLiteAgent' / 'code_safety.jinja2'}, 'interaction_polish': {'response_model': VegaLiteSpecUpdate, 'template': PROMPTS_DIR / 'VegaLiteAgent' / 'interaction_polish.jinja2'}, 'annotate_plot': {'response_model': VegaLiteSpecUpdate, 'template': PROMPTS_DIR / 'VegaLiteAgent' / 'annotate_plot.jinja2'}, 'revise_output': {'response_model': RetrySpec, 'template': PROMPTS_DIR / 'VegaLiteAgent' / 'revise_output.jinja2'}}) class-attribute instance-attribute

purpose = param.String(default='Generates one or more vega-lite plot specifications from the input data pipeline.') class-attribute instance-attribute

reserve_base_form = param.Boolean(default=True, doc='\n Guarantee one retrieved section is the base form of a chart type, when\n the candidate pool contains one. A bare request like "plot a bar chart"\n matches every bar variant about equally -- similarity across the top\n candidates spans less than the noise floor -- so the plain section loses\n to specialisations that simply have more text to match on. Sections are\n flagged deterministically at build time from their title, which is why\n this reserves a slot rather than nudging a score.') class-attribute instance-attribute

user = param.String(default='Vega') class-attribute instance-attribute

vector_store_path = param.Path(default=None, check_exists=False, doc='\n Path to a custom vector store for storing and retrieving Vega-Lite examples;\n if not provided a default store will be used depending on the LLM--\n OpenAIEmbeddings for OpenAI LLM or NumpyEmbeddings for all others.') class-attribute instance-attribute

view_type = VegaLiteView class-attribute instance-attribute

annotate(instruction, messages, context, spec, view=None) async

Apply annotations based on user request.

Parameters:

Name Type Description Default
instruction str

User's description of what to annotate

required
messages list[Message]

Chat history for context

required
context TContext

Session context

required
spec dict

The current VegaLite specification (full dict with 'spec' key)

required
view VegaLiteEditor | None

Optional VegaLite editor to export plot image from for vision analysis.

None

Returns:

Type Description
str

Updated specification with annotations

respond(messages, context, step_title=None) async

Generates one or more VegaLite visualizations using a progressive building approach with real-time updates.

revise(feedback, messages, context, view=None, spec=None, language=None, errors=None, **kwargs) async

Revise a VegaLite specification based on user feedback.

This override adds: 1. Doc examples from vector store 2. Plot image for vision analysis (if view is a VegaLiteEditor)

Parameters:

Name Type Description Default
feedback str

User's feedback or instruction for revision.

required
messages list[Message]

Chat message history.

required
context TContext

Session context.

required
view LumenEditor | None

The editor containing the current spec to revise.

None
spec str | None

The spec string (used if view is None).

None
language str | None

The spec language (used if view is None).

None
errors list[str] | None

List of errors to include in context.

None
**kwargs

Additional arguments passed to parent revise.

{}

Returns:

Type Description
str

The revised YAML specification.

XArraySQLSource

Bases: BaseSQLSource

SQL-queryable xarray data source for Lumen.

Wraps xarray datasets with Apache DataFusion (via xarray-sql) to provide full SQL query support. Each data variable in the dataset is exposed as a separate SQL table with coordinate columns.

To use this source, install the optional xarray dependencies::

pip install lumen[xarray]

Parameters:

Name Type Description Default
uri str

Path or URL to a NetCDF/Zarr/HDF5/GRIB file.

required
engine str or None

xarray engine to use. Auto-detected from file extension if None.

required
chunks dict or str or None

Chunking specification for dask-backed lazy loading. Use 'auto' for automatic chunking, or a dict like {'time': 100}.

required
open_kwargs dict

Additional keyword arguments passed to xr.open_dataset().

required
variables list or None

Subset of data variables to expose. None means all variables.

required

Examples:

>>> source = XArraySQLSource(uri="air_temperature.nc")
>>> source.get_tables()
['air']
>>> df = source.execute("SELECT lat, lon, AVG(air) FROM air GROUP BY lat, lon")
>>> schema = source.get_schema("air")

chunks = param.Parameter(default='auto', doc="\n Dask chunk specification for lazy loading.\n Use 'auto', a dict like {'time': 100}, or None for eager loading.") class-attribute instance-attribute

dataset property

Access the underlying xarray Dataset.

engine = param.String(default=None, allow_None=True, doc='\n xarray backend engine. Auto-detected from file extension if None.\n Options: netcdf4, zarr, cfgrib, h5netcdf.') class-attribute instance-attribute

open_kwargs = param.Dict(default={}, doc='\n Additional keyword arguments for xr.open_dataset().') class-attribute instance-attribute

source_type = 'xarray' class-attribute

sql_expr = param.String(default='SELECT * FROM {table}', doc='\n The SQL expression template for table queries.') class-attribute instance-attribute

tables = param.ClassSelector(class_=(list, dict), default=None, allow_None=True, doc='\n List or dict of tables. Populated automatically from dataset variables.\n When a dict, maps logical table names to SQL expressions.') class-attribute instance-attribute

uri = param.String(default=None, allow_None=True, doc='\n Path or URL to the xarray-compatible data file.\n Supports NetCDF (.nc), Zarr (.zarr), HDF5 (.h5), GRIB (.grib).') class-attribute instance-attribute

variables = param.List(default=None, allow_None=True, doc='\n Subset of data variables to expose as tables.\n None means all variables are exposed.') class-attribute instance-attribute

close()

Clean up resources.

create_sql_expr_source(tables, params=None, **kwargs)

Create a new XArraySQLSource sharing the same DataFusion context.

execute(sql_query, params=None, *args, **kwargs)

Execute a SQL query against the DataFusion context.

fetch(sql_query, params=None)

Fetch query results in dataframe_backend, built by DataFusion.

DataFusion is arrow native, so the to_pandas in execute is a conversion this can skip entirely when another library was asked for.

from_dataset(dataset, chunks='auto', **kwargs) classmethod

Creates an in-memory XArraySQLSource from an xarray Dataset.

Arguments

dataset: xr.Dataset The xarray Dataset to query. chunks: dict or str Chunk specification for DataFusion registration. kwargs: any Additional keyword arguments for the source.

Returns:

Name Type Description
source XArraySQLSource

get(table, **query)

get_schema(table=None, limit=None, shuffle=False)

get_tables()

normalize_table(table)

to_dataset(table, **query)

Return the query result as a gridded xr.Dataset.

Uses xarray-sql's to_dataset, which is lazy and keeps the data gridded, so a gridded source is never materialized as long-form.

_keyless_options_message()

Tell a user with no provider configured what works without an API key on this machine.

content_to_text(content)

Extract the text portion from a message content field.

Handles both plain string content and multimodal content lists (e.g. containing both text strings and image objects). Returns just the concatenated text parts.

Parameters:

Name Type Description Default
content str | list

The message content — either a plain string or a list of mixed content items (strings, Image objects, etc.).

required

Returns:

Type Description
str

The extracted text content.

export_notebook(outputs, preamble='')

format_msg_content(content)

Format a message content field for logging.

Handles plain strings, multimodal lists (text + images), and arbitrary objects. Non-string items are represented as compact Type[id] placeholders so that base64 blobs never flood logs.

Parameters:

Name Type Description Default
content Any

The message content — a string, a list of mixed items, or any object.

required

Returns:

Type Description
str

A human-readable, log-safe representation of the content.

get_available_llm()

Detect an available LLM provider: the first one whose API key environment variable is set.

Providers that need no key, such as Ollama and the coding CLIs, are only used when requested explicitly.

Returns:

Type Description
type[Llm] | None

The LLM class, or None if no provider is available.

local_cli_providers()

Coding CLI providers installed on PATH, mapped to whether they report a signed-in user.

log_debug(msg, offset=24, prefix='', suffix='', show_sep=None, show_length=False)

Log a debug message with a separator line above and below.

normalize_table_name(name)

Normalize a table name to a valid SQL identifier.

Replaces all non-word characters (anything except letters, digits, and underscores) with underscores, strips leading/trailing underscores, and converts to lowercase.

This matches the behavior of DuckDBSource.normalize_table() to ensure consistent table naming across the codebase.

Parameters:

Name Type Description Default
name str

The table name to normalize (e.g., filename without extension)

required

Returns:

Type Description
str

The normalized table name

Examples:

>>> normalize_table_name("customer (orders)")
'customer_orders'
>>> normalize_table_name("My-Data File")
'my_data_file'
>>> normalize_table_name("table__name")
'table_name'

read_geo_file(file_obj, extension, alias)

Parse a geospatial file, returning WKB geometry + conversion SQL.

try_import_xarray(load=True)

Import and return xarray, or None if xarray or xarray-sql is unavailable.

wrap_logfire(span_name=None, extract_args=True, **instrument_kwargs)

Decorator to instrument a function with logfire if llm.logfire_tags is provided. Expects the decorated function to have an 'llm' attribute (on self) or be passed via instrument_kwargs.