User Interface¶
The ExplorerUI application and its building blocks.
lumen.ai.ui
¶
CODE_EXEC_WARNING = '\n### ⚠️ Security Warning\n\nEnabling code execution allows the AI to generate and run Python code on your system.\n\n**This poses significant security risks**, especially in environments with:\n\n- API keys or secrets\n- Database credentials\n- Access to sensitive files or systems\n- Network access to internal services\n\nLLM-generated code may be unpredictable. While validation layers reduce accidental errors, they **do not provide meaningful protection** against malicious or compromised inputs.\n'
module-attribute
¶
DATA_SOURCES_HELP = '\n**Add data to explore with Lumen AI.**\n\n**Supported formats:**\n\n- CSV, Parquet, JSON, Excel, GeoJSON files\n- Remote URLs (https://example.com/data.csv)\n- Databases via SQLAlchemy\n\n**How to add data:**\n\n1. **Upload Data** — Drag and drop files or click to browse\n2. **Fetch Remote Data** — Enter URLs to fetch remote data\n\n**Tips:**\n\n- Upload .md or .txt files alongside data to give the AI context\n- Multiple sources can be used together in queries\n- Data connections persist for your session\n'
module-attribute
¶
DEMO_MESSAGES = ['What datasets are available?', 'Show me the the first dataset and its columns.', 'What are some interesting analyses I can do?', 'Perform an analysis using the first suggestion.', 'Show me a plot of these results.']
module-attribute
¶
DataT = str | Path | Source | Pipeline
module-attribute
¶
EXPLORATIONS_INTRO_HELP = '**Select a table below** to start a new exploration. Click on Help on the left for more info.'
module-attribute
¶
EXPLORATION_CAPTION = '\nLaunch new explorations by chatting or selecting a table.\n\nAsk follow-up questions by navigating to an existing exploration.\n'
module-attribute
¶
EXPLORATION_TOOLBAR_HELP = '\nToolbar actions for this exploration:\n\n- **Export Output** — Download as CSV/Excel (tables) or PNG/SVG (visualizations)\n- **Copy** — Copy the spec to clipboard\n- **Sparkle** — Ask AI to revise the output\n- **Annotations** — Add highlights to visualizations (plots only)\n- **Pop-out** — Move to a separate pane\n\nDrag the divider between editor and view to resize.\n'
module-attribute
¶
EXPLORATION_VIEW_HELP = 'Use < > to expand/collapse panels. Edit the spec (top) and the view (bottom) syncs. Click ✨ to ask LLM to revise.'
module-attribute
¶
GETTING_STARTED_SUGGESTIONS = [('search', 'What data is available?'), ('description', 'Show me a dataset'), ('bar_chart', 'Visualize the data')]
module-attribute
¶
HELP_EDITOR = "\n**📝 Two-pane editor:**\n\nWhen you ask a question, you'll see two panes:\n\n- **Top pane** = the spec (SQL query or YAML configuration)\n - Edit directly here to modify the analysis\n - Changes are applied in real-time to the preview below\n - Great for small tweaks like changing colors or sort order\n\n- **Bottom pane** = live preview\n - Automatically updates when you change the spec\n - Shows tables, charts, or other visualizations\n"
module-attribute
¶
HELP_EXPLORATIONS = '\n**🔍 What are Explorations?**\n\nAn Exploration is a persistent, contextual workspace for working with a specific dataset. It is created when a SQL query is generated and captures the full state of the interaction, including the conversation, generated analyses, visualizations, and other data artifacts.\n\nExplorations can evolve over time, support multiple questions on the same data, and can be revisited or exported as a coherent unit.\n\nExplorations start from the global context (available sources and metadata). If a question is a follow-up, the new exploration is nested under the parent; if it is not, Lumen creates a new top-level exploration.\n\n**🧭 Navigation menu (left panel):**\n- Shows all explorations in a persistent tree\n- Click an exploration to switch context\n- Follow-up explorations appear nested under their parent\n- Use the item menu (⋮) to remove an exploration\n'
module-attribute
¶
HELP_EXPORT = '\n**📄 Export individual results:**\n\n- **Export Output** ▼ (top of each result) - Download specific results:\n - **Tables**: SQL, CSV, or Excel formats\n - **Visualizations**: PNG, JPEG, PDF, SVG, or HTML\n\n**💾 Save your work:**\n\n- Use **Export Notebook** in the navigation menu to download everything as a Jupyter notebook\n- Includes all SQL queries, visualizations, and results\n- Perfect for documentation or sharing with colleagues\n- Each exploration saves automatically during your session\n\n**📋 Report Mode:**\n\n- Click **Report** in the left sidebar\n- View all explorations on one page\n- Export everything as a single notebook\n'
module-attribute
¶
HELP_GETTING_STARTED = '\nAsk questions in plain English and Lumen AI automatically generates SQL queries and creates visualizations.\n\n**🚀 Start exploring:**\n\n- Type a question like "What datasets are available?"\n- Click the quick action buttons below the chat\n- Select a table and click **Explore** to start analyzing\n\n**❓ Example questions:**\n\n**Basic queries:**\n- "Show me the top 10 rows"\n- "What are the columns in the dataset?"\n- "Give me a summary of the data"\n\n**Visualizations:**\n- "What\'s the average revenue by region? Plot as a bar chart"\n- "Create a scatter plot of X vs Y, colored by category"\n- "Show me a histogram of sales with 20 bins"\n\n**Complex queries:**\n- "Filter for records over $1000 and show the distribution by category"\n- "Group by department and calculate average salary. Then plot the result."\n- "Are there any outliers in the data?"\n'
module-attribute
¶
HELP_INTERFACE = '\n**🖥️ Panel Controls:**\n- **< >** buttons - Collapse/expand the chat and results panels\n- **^ v** arrows - Expand/collapse individual analyses\n\n**📂 Sidebar:**\n- **Explore** - Main chat and analysis mode (default)\n- **Report** - View all explorations on one page\n- **Sources** - Add or manage your data\n- **Settings** - Control AI behavior settings and LLM configuration\n- **Help** - Open this help dialog\n\n**⌨️ Keyboard Shortcuts:**\n- **Enter** - Send message\n- **Shift + Enter** - New line in chat\n'
module-attribute
¶
HELP_NO_SOURCES = '**Welcome to Lumen AI!**\n\n⚠️ **No data sources connected yet.**\n\n**To get started:**\n\n1. Click **Data Sources** in the left sidebar\n2. Upload a file (CSV, Parquet, JSON, Excel, GeoJSON) or connect to a database\n3. Once your data is loaded, come back here to start asking questions!\n\nYou can also drag & drop files directly into the chat interface.\n\n**Once you have data, you can ask questions like:**\n\n*Basic queries:*\n- "Show me the top 10 rows"\n- "What are the columns in the dataset?"\n- "Give me a summary of the data"\n\n*Visualizations:*\n- "What\'s the average revenue by region? Plot as a bar chart"\n- "Create a scatter plot of X vs Y, colored by category"\n\n*Complex queries:*\n- "Filter for records over $1000 and show the distribution by category"\n- "Group by department and calculate average salary. Then plot the result."\n\n---\n\n➡️ After adding data, click **Navigation** above to learn how to move around the interface.'
module-attribute
¶
HELP_RESULTS = '\n**🛠️ Toolbar actions (top of each result):**\n\n- **📋** - Copy YAML spec to clipboard\n- **✨** - Ask the LLM to revise this analysis\n- **💬** - Add highlights or callouts to visualizations (plots only)\n- **↗** - Pop out into a separate pane for comparison\n\n**📊 Table features:**\n- Click **column headers** to sort (^ = ascending, v = descending)\n- Use **pagination** (< >, page numbers) at the bottom to navigate large datasets\n- All data is interactive - you can explore it directly\n\n**🔄 Refining results:**\n- **Rerun** (⟲ in the chat input + menu) - Re-execute the last query if there was an error\n- **Undo** (↶ in the chat input + menu) - Remove the last response and try again\n- **Continue the conversation** - Send a new message to refine: "Can you make that chart show only the top 5?"\n- **Manual editing** - Edit the spec directly for precise control\n'
module-attribute
¶
HELP_TIPS = '\n**🔗 Combine multiple requests:**\nYou can ask the AI to perform several steps at once.\n\n- "Filter the data for 2023, create a bar chart of sales by region, and then summarize the top 3 regions."\n\n**💡 Ask for suggestions:**\nNot sure where to start? Ask:\n- "What could be interesting to explore in this data?"\n- "Give me 3 ideas for visualizations."\n\n**✨ Refining results:**\nYou don\'t have to get it right the first time.\n- "Can you make that chart show only the top 5?"\n- "Change the color of the bars to green."\n- "Add a trend line to the scatter plot."\n\n**🗂️ Need more data?**\nClick **Sources** in the left sidebar to add files or databases at any time.\n\n**🧠 Chain of Thought:**\nEnable **Chain of Thought** in Settings to see the AI\'s step-by-step reasoning.\n- This is great for debugging or understanding complex queries.\n\n---\n\n✔️ You\'re all set! Close this dialog and start exploring.\n'
module-attribute
¶
IMAGE_MIME_TYPES = {'.png': 'image/png', '.jpg': 'image/jpeg', '.jpeg': 'image/jpeg', '.gif': 'image/gif', '.webp': 'image/webp', '.svg': 'image/svg+xml', '.bmp': 'image/bmp'}
module-attribute
¶
LOGO = (Path(__file__).parent.parent / '_assets' / 'logo.svg').read_text()
module-attribute
¶
PAGE_SX = {'.sidebar': {'transition': 'width 0.2s ease-in-out'}, '.sidebar:hover': {'width': '140px', 'transitionDelay': '0.5s'}, '&.mui-light .sidebar': {'bgcolor': 'var(--mui-palette-grey-50)'}}
module-attribute
¶
PREFERENCES_HELP = "\n**Control how Lumen AI analyzes your questions.**\n\n**Chain of Thought** — Show the AI's reasoning steps as it works\n\n- Useful for understanding how the AI reached an answer\n- Displays expandable cards with step-by-step logic\n- Disabled by default\n\n**SQL Planning** — Run discovery queries before generating final SQL\n\n- Improves accuracy for complex questions\n- The AI explores your schema before querying\n- Disabled by default\n\n**Validation Step** — Double-check if the response answered your question\n\n- Catches data errors early\n- Verifies results match your intent\n- Enabled by default\n\n**Configure AI Models** — Choose which LLM provider and models to use for different tasks\n"
module-attribute
¶
PROVIDED_SOURCE_NAME = 'ProvidedSource00000'
module-attribute
¶
REPORT_CAPTION = 'Use ▶ to execute all, × to clear outputs, ∨∧ to collapse/expand sections. Click exploration names to jump to them.'
module-attribute
¶
REPORT_HELP = '\n**Report Mode: View All Your Work**\n\nReport mode displays all explorations in one scrollable page for review and presentation.\n\n**Page Controls:**\n\n- **▶** (Execute Report) - Run all analyses and generate fresh results\n- **×** (Clear outputs) - Remove all output displays to start fresh\n- **∨ ∧** (Collapse/Expand) - Show or hide all analysis sections at once\n\n**Navigation:**\n\n- Click an exploration name in the left sidebar to scroll to it\n- All analyses appear in order with their results\n- Scroll through the entire report or jump to specific sections\n\n**Export:**\n\n- Use **Export Notebook** in the navigation menu to download everything as one Jupyter notebook\n- Includes all SQL queries, visualizations, and results\n- Perfect for sharing findings or creating documentation\n\n**To get back:**\n\n- Click **Explore** in the left sidebar to return to chat mode\n- Or click a specific exploration in the navigation menu to work on it\n'
module-attribute
¶
SOURCE_TABLE_SEPARATOR = ' ⦙ '
module-attribute
¶
SPLASH_HELP_HINT = 'Click on Help on the left for more info.'
module-attribute
¶
TContext = Mapping[str, Any]
module-attribute
¶
XARRAY_EXTENSIONS = ('nc', 'nc4', 'netcdf', 'h5', 'hdf5', 'he5', 'zarr', 'grib', 'grib2', 'grb', 'grb2')
module-attribute
¶
log = getLogger(__name__)
module-attribute
¶
ActorTask
¶
Bases:
An ActorTask wraps an Actor in a way that allows persisting
the message history, instructions, and context as well as making
it possible to track its outputs.
actor = param.ClassSelector(class_=Actor)
class-attribute
instance-attribute
¶
input_schema
property
¶
level = 3
class-attribute
instance-attribute
¶
output_schema
property
¶
editor(show_title=True)
¶
Returns the editor for the tasks.
Arguments
show_title: bool Whether to show the title of the tasks.
Returns:
| Type | Description |
|---|---|
The editor for the tasks.
|
|
prepare(context)
async
¶
AnalysisAgent
¶
Bases:
analyses = param.List([])
class-attribute
instance-attribute
¶
conditions = param.List(default=['Use for custom analysis, advanced analytics, or domain-specific methods', "Use when the user query matches one of the available analyses' name or description below.", "Include the selected analysis' required cols in the instructions", 'NOT for simple queries or basic visualizations'])
class-attribute
instance-attribute
¶
input_schema = AnalysisInputs
class-attribute
instance-attribute
¶
output_schema = AnalysisOutputs
class-attribute
instance-attribute
¶
prompts = param.Dict(default={'main': {'template': PROMPTS_DIR / 'AnalysisAgent' / 'main.jinja2', 'response_model': make_analysis_model}})
class-attribute
instance-attribute
¶
purpose = param.String(default='Perform custom analyses that are reliable and repeatable.')
class-attribute
instance-attribute
¶
user = param.String(default='Analysis')
class-attribute
instance-attribute
¶
respond(messages, context, step_title=None)
async
¶
AnalysisOutput
¶
BaseCodeAgent
¶
Bases:
Base class for view agents that can generate and execute code.
Subclasses must:
- Set _executor_class to the appropriate CodeExecutor subclass
- Implement _generate_code_spec() for their specific code generation flow
code_execution = param.Selector(default='disabled', objects=['disabled', 'prompt', 'llm', 'allow'], doc="\n Code execution mode for generating visualizations via code:\n - disabled: No code execution; generate declarative specs only (safe for production)\n - prompt: Generate code, prompt user for permission to execute\n - llm: Generate code, validate with LLM safety check, then execute\n - allow: Generate and execute code without user confirmation\n\n ⚠️ WARNING: The 'prompt', 'llm', and 'allow' modes execute LLM-generated code and\n must NEVER be enabled in production environments with access to secrets, credentials,\n or sensitive data.\n ", allow_refs=True)
class-attribute
instance-attribute
¶
code_execution_enabled
property
¶
Whether code execution is enabled (any mode except 'disabled').
BaseSourceControls
¶
Bases:
Base class for source controls with lifecycle management.
Provides progress reporting, message display, load-button wiring,
and source-output registration. Subclasses implement _load()
or call _run_load() manually for custom trigger patterns.
cancel = param.Event(doc='Cancel')
class-attribute
instance-attribute
¶
context = param.Dict(default={})
class-attribute
instance-attribute
¶
disabled = param.Boolean(default=False, doc='Disable controls')
class-attribute
instance-attribute
¶
label = param.String(default='', constant=True, doc='\n HTML label shown in the sidebar for this control.')
class-attribute
instance-attribute
¶
load = param.Event(doc='Trigger data loading')
class-attribute
instance-attribute
¶
load_button_icon = param.String(default='download', doc='\n Material icon name for the load button.')
class-attribute
instance-attribute
¶
load_button_label = param.String(default='Load Data', doc='\n Text label for the load button.')
class-attribute
instance-attribute
¶
load_mode = param.Selector(default='button', objects=['button', 'manual'], constant=True, doc='\n How data loading is triggered:\n - "button": Show load button, clicking triggers _load()\n - "manual": No button, use _run_load(coro) for custom triggers\n ')
class-attribute
instance-attribute
¶
outputs = param.Dict(default={})
class-attribute
instance-attribute
¶
source_catalog = param.Parameter(default=None, doc='Reference to SourceCatalog instance')
class-attribute
instance-attribute
¶
source_name_prefix = param.String(default='UploadedSource', doc='\n Prefix for auto-generated source names.')
class-attribute
instance-attribute
¶
upload_successful = param.Event(doc='Triggered when files are successfully uploaded and processed')
class-attribute
instance-attribute
¶
ChatAgent
¶
Bases:
ChatAgent provides general information about available data and other topics to the user. When data is available, it acts as an analyst providing insights and interpretations.
conditions = param.List(default=["Use for general conversation that doesn't require fetching or querying data", 'Use for technical questions about programming, functions, methods, libraries, or APIs', "Use when user asks to 'explain', 'interpret', 'analyze', 'summarize', or 'comment on' existing data in context", "NOT when user asks to 'query', 'filter', 'calculate', 'aggregate', or 'transform' data", 'NOT for creating new data transformations - only for explaining data that already exists'])
class-attribute
instance-attribute
¶
output_schema = ChatOutputs
class-attribute
instance-attribute
¶
prompts = param.Dict(default={'main': {'template': PROMPTS_DIR / 'ChatAgent' / 'main.jinja2'}})
class-attribute
instance-attribute
¶
purpose = param.String(default='\n Provides conversational assistance and interprets existing results.\n Handles general questions, technical documentation, and programming help.\n When data has been retrieved, explains findings in accessible terms.')
class-attribute
instance-attribute
¶
respond(messages, context, step_title=None)
async
¶
ChatLogs
¶
Bases:
conn = sqlite3.connect(self.filename)
instance-attribute
¶
cursor = self.conn.cursor()
instance-attribute
¶
filename = param.String(default='chat_logs.db')
class-attribute
instance-attribute
¶
update_status(message_id, liked=None, disliked=None, removed=None)
¶
upsert(session_id, message_id, message_index, message_user, message_content)
¶
ChatUI
¶
Bases:
ChatUI provides a high-level entrypoint to start chatting with your data in a chat based UI.
This high-level wrapper allows providing the data sources you will be chatting with and then configures the assistant and agents.
:Example:
.. code-block:: python
import lumen.ai as lmai
lmai.ChatUI(data='~/data.csv').servable()
llm_choices = param.List(default=[], doc='\n List of available LLM model choices to show in the configuration dialog.')
class-attribute
instance-attribute
¶
provider_choices = param.Dict(default={}, doc='\n Available LLM providers to show in the configuration dialog.')
class-attribute
instance-attribute
¶
title = param.String(default='Lumen ChatUI', doc='Title of the app.')
class-attribute
instance-attribute
¶
Coordinator
¶
Bases: ,
A Coordinator is responsible for coordinating the actions of a number of agents towards the user defined query by computing an execution graph and then executing each step along the graph.
agents = param.List(default=[ChatAgent], doc='\n List of agents to coordinate.')
class-attribute
instance-attribute
¶
context = param.Dict(default={})
class-attribute
instance-attribute
¶
history = param.Integer(default=3, doc='\n Number of previous user-assistant interactions to include in the chat history.')
class-attribute
instance-attribute
¶
llm_tools = param.List(default=[make_load_metaset_relevant_docs_tool, make_document_vector_llm_tools, make_monty_llm_tool], doc='\n List of tools for the Planner to make available to the LLM. The tools are also\n made available to the agents.')
class-attribute
instance-attribute
¶
prompts = param.Dict(default={'main': {}, 'tool_relevance': {'template': PROMPTS_DIR / 'Coordinator' / 'tool_relevance.jinja2', 'response_model': ThinkingYesNo}})
class-attribute
instance-attribute
¶
validation_enabled = param.Boolean(default=False, allow_refs=True, doc='\n Whether to enable the ValidationAgent in the planning process.')
class-attribute
instance-attribute
¶
verbose = param.Boolean(default=False, allow_refs=True, doc='\n Whether to show verbose output.')
class-attribute
instance-attribute
¶
within_ui = param.Boolean(default=False, constant=True, doc='\n Whether this coordinator is being used within the UI.')
class-attribute
instance-attribute
¶
respond(messages, context, **kwargs)
async
¶
sync(context)
async
¶
DeckGLAgent
¶
Bases:
Agent for generating DeckGL 3D map visualizations.
Supports two generation modes: - When code_execution is 'disabled': Generate DeckGL JSON specs directly - When code_execution is enabled: Generate PyDeck Python code, execute safely, convert to spec
conditions = param.List(default=['Use for 3D geographic visualizations, map-based data, or when user requests DeckGL/deck.gl', 'Use for large-scale geospatial data with latitude/longitude coordinates', 'Use for hexbin aggregations, heatmaps, or 3D extruded visualizations on maps', "Can render a geometry column (GeoDataFrame polygons/lines/points) on a basemap, suited to 3D/extruded maps or geometry whose schema is 'geographic' (a lat/lon CRS) so the basemap gives real-world context", 'NOT when the table only names places (a state or country column) without geometry or lat/lon, since deck.gl has nothing to draw and cannot fetch boundaries'])
class-attribute
instance-attribute
¶
default_map_style = 'https://basemaps.cartocdn.com/gl/dark-matter-gl-style/style.json'
class-attribute
instance-attribute
¶
prompts = param.Dict(default={'main': {'response_model': DeckGLSpec, 'template': PROMPTS_DIR / 'DeckGLAgent' / 'main.jinja2'}, 'main_pydeck': {'response_model': PyDeckSpec, 'template': PROMPTS_DIR / 'DeckGLAgent' / 'main_pydeck.jinja2'}, 'code_safety': {'response_model': CodeSafetyCheck, 'template': PROMPTS_DIR / 'DeckGLAgent' / 'code_safety.jinja2'}, 'revise_output': {'response_model': RetrySpec, 'template': PROMPTS_DIR / 'DeckGLAgent' / 'revise_output.jinja2'}})
class-attribute
instance-attribute
¶
purpose = param.String(default='Generates DeckGL 3D map visualizations from geographic data.')
class-attribute
instance-attribute
¶
user = param.String(default='DeckGL')
class-attribute
instance-attribute
¶
view_type = DeckGLView
class-attribute
instance-attribute
¶
respond(messages, context, step_title=None)
async
¶
Generate a DeckGL visualization.
DocumentListAgent
¶
Bases:
The DocumentListAgent lists all available documents provided by the user.
conditions = param.List(default=['Use when user asks to list or see all available documents', 'NOT when user asks about specific document content'])
class-attribute
instance-attribute
¶
input_schema = DocumentListInputs
class-attribute
instance-attribute
¶
purpose = param.String(default='\n Displays a list of all available documents.')
class-attribute
instance-attribute
¶
user = param.String(default='Documents')
class-attribute
instance-attribute
¶
applies(context)
async
classmethod
¶
DocumentSummarizerAgent
¶
Bases:
Synthesizes relevant document context into a user-focused summary.
conditions = param.List(default=['Use when user asks questions answerable from documentation or uploaded metadata files', 'Use for summarization of documentation', 'NOT when user only asks to list available documents'])
class-attribute
instance-attribute
¶
input_schema = DocumentSummarizerInputs
class-attribute
instance-attribute
¶
not_with = param.List(default=['DbtslAgent', 'SQLAgent'])
class-attribute
instance-attribute
¶
output_schema = DocumentSummarizerOutputs
class-attribute
instance-attribute
¶
prompts = param.Dict(default={'main': {'template': PROMPTS_DIR / 'DocumentSummarizerAgent' / 'main.jinja2'}})
class-attribute
instance-attribute
¶
purpose = param.String(default='\n Synthesizes relevant documents into an answer focused on the user query.\n Also renders a document editor with tabbed previews of matched files.')
class-attribute
instance-attribute
¶
user = param.String(default='Summarizer')
class-attribute
instance-attribute
¶
applies(context)
async
classmethod
¶
respond(messages, context, step_title=None)
async
¶
DownloadSourceControls
¶
Bases:
Controls for downloading files from URLs and extracting tabular data.
Supports CSV, JSON, Parquet, Excel, and HTML files. For HTML pages, all tables are extracted and loaded as separate database tables.
Kaggle dataset URLs (e.g. https://www.kaggle.com/datasets/owner/name)
are also supported and will download all data files in the dataset into
a single source with one table per file.
download_url = param.String(default='', label='Download URL(s)', doc='Enter one or more URLs, one per line, and press <Shift+Enter> to download')
class-attribute
instance-attribute
¶
input_placeholder = param.String(default='Enter URLs, one per line, and press <Shift+Enter> to download')
class-attribute
instance-attribute
¶
label = '<span class="material-icons" style="vertical-align: middle;">download</span> Fetch Remote Data'
class-attribute
instance-attribute
¶
load_mode = 'manual'
class-attribute
instance-attribute
¶
source_name_prefix = 'DownloadedSource'
class-attribute
instance-attribute
¶
as_tools(query=None, top_k=5)
¶
Return a tool that allows the agent to download data from URLs.
The tool accepts a URL string and downloads the file, extracting tabular data from CSV, JSON, Parquet, Excel, or HTML files.
DuckDBSource
¶
Bases:
DuckDBSource provides a simple wrapper around the DuckDB SQL connector.
To specify tables to be queried provide a list or dictionary of
tables. A SQL expression to fetch the data from the table will
then be generated using the sql_expr, e.g. if we specify a local
table flights.db the default sql_expr SELECT * FROM {table} will
expand that to SELECT * FROM flights.db. If you want to specify
a full SQL expression as a table you must change the sql_expr to
'{table}' ensuring no further templating is applied.
Note that certain functionality in DuckDB requires modules to be
loaded before making a query. These can be specified using the
initializers parameter, providing the ability to define DuckDb
statements to be run when initializing the connection.
connection
property
¶
dialect = 'duckdb'
class-attribute
instance-attribute
¶
ephemeral = param.Boolean(default=False, doc='\n Whether the data is ephemeral, i.e. manually inserted into the\n DuckDB table or derived from real data.')
class-attribute
instance-attribute
¶
filter_in_sql = param.Boolean(default=True, doc='\n Whether to apply filters in SQL or in-memory.')
class-attribute
instance-attribute
¶
geometry_crs = param.String(default=None, allow_None=True, doc='\n CRS to reapply to geometry columns after the WKB roundtrip through\n DuckDB, used as a fallback when the column type carries no CRS of its\n own. May be set explicitly for a known dataset; an ingested frame\n instead records its CRS on the GEOMETRY column type itself.')
class-attribute
instance-attribute
¶
initializers = param.List(default=[], doc='\n SQL statements to run to initialize the connection.')
class-attribute
instance-attribute
¶
load_schema = param.Boolean(default=True, doc='Whether to load the schema')
class-attribute
instance-attribute
¶
mirrors = param.Dict(default={}, doc='\n Mirrors the tables into the DuckDB database. The mirrors\n should define a mapping from the table names to the source of\n the mirror which may be defined as a Pipeline or a tuple of\n the source and the table.')
class-attribute
instance-attribute
¶
read_only = param.Boolean(default=None, doc='\n Whether to open the DuckDB database in read-only mode.')
class-attribute
instance-attribute
¶
source_type = 'duckdb'
class-attribute
instance-attribute
¶
sql_expr = param.String(default='SELECT * FROM {table}', doc='\n The SQL expression to execute.')
class-attribute
instance-attribute
¶
table_params = param.Dict(default={}, doc='\n Dictionary mapping table names to lists of SQL parameters.\n Parameters are used with placeholders (?) in SQL expressions.')
class-attribute
instance-attribute
¶
tables = param.ClassSelector(class_=(list, dict), doc='\n List or dictionary of tables.')
class-attribute
instance-attribute
¶
uri = param.String(doc='The URI of the DuckDB database')
class-attribute
instance-attribute
¶
close()
¶
Close the DuckDB connection, releasing associated resources.
This method should be called when the source is no longer needed to prevent connection leaks and properly clean up server-side resources.
create_sql_expr_source(tables, materialize=True, params=None, **kwargs)
¶
Creates a new SQL Source given a set of table names and corresponding SQL expressions.
Arguments
tables: dict[str, str] Mapping from table name to SQL expression. materialize: bool Whether to materialize new tables params: dict[str, list | dict] | None Optional mapping from table name to parameters: - list: Positional parameters for ? placeholders - dict: Named parameters for $param_name placeholders kwargs: any Additional keyword arguments.
Returns:
| Name | Type | Description |
|---|---|---|
source |
|
|
execute(sql_query, params=None, *args, **kwargs)
¶
execute_with_timeout(sql_query, timeout, fetch=False)
async
¶
fetch(sql_query, params=None)
¶
Fetch query results in dataframe_backend, built by DuckDB itself.
Not the inherited fetch, which reads through execute. Two reasons: the data path asks for date_as_object so a DATE stays a date rather than becoming a timestamp, and DuckDB can materialise polars and arrow directly. Building the frame in the asked-for library rather than converting a pandas one is most of the point of the parameter: on a 5M row table fetching arrow rather than pandas is an order of magnitude quicker and allocates less than half the memory.
from_df(tables, **kwargs)
classmethod
¶
Creates an ephemeral, in-memory DuckDBSource containing the supplied dataframe.
Arguments
tables: dict[str, pandas.DataFrame] A dictionary mapping from table names to DataFrames kwargs: any Additional keyword arguments for the source
Returns:
| Name | Type | Description |
|---|---|---|
source |
|
|
from_spec(spec)
classmethod
¶
get(table, **query)
¶
get_tables()
¶
missing_table(error)
¶
normalize_table(table)
¶
to_spec(context=None)
¶
ErrorDescription
¶
Bases:
Represents a user-facing error explanation.
Style rules: - Do not use first-person pronouns (“I”, “we”). - All outputs must be written in neutral, system-level language.
Guidance: - If you think it's an internal error, indicate that the user should rerun. - Otherwise, ask the user to either replan OR clarify their questions. - You are given the error type, message, the user query, and the plan the planning agent came up with.
explanation = Field(description='A brief description of the error suitable for a non-technical user.', max_length=300)
class-attribute
instance-attribute
¶
Exploration
¶
Bases:
context = param.Dict(allow_refs=True)
class-attribute
instance-attribute
¶
conversation = Children()
class-attribute
instance-attribute
¶
initialized = param.Boolean(default=False)
class-attribute
instance-attribute
¶
parent = param.Parameter()
class-attribute
instance-attribute
¶
plan = param.ClassSelector(class_=Plan)
class-attribute
instance-attribute
¶
subtitle = param.String(default='')
class-attribute
instance-attribute
¶
title = param.String(default='')
class-attribute
instance-attribute
¶
view = Child()
class-attribute
instance-attribute
¶
ExplorerUI
¶
Bases:
ExplorerUI provides a high-level entrypoint to start chatting with your data in split UI allowing users to load tables, explore them using Graphic Walker, and then interrogate the data via a chat interface.
This high-level wrapper allows providing the data sources you will be chatting with and then configures the assistant and agents.
:Example:
.. code-block:: python
import lumen.ai as lmai
lmai.ExplorerUI(data='~/data.csv').servable()
code_execution = param.Selector(default='hidden', objects=['hidden', 'disabled', 'prompt', 'llm', 'allow'], doc="\n Code execution mode for generating Vega-Lite specs via Altair code.\n Controls whether the code execution selector appears in the UI preferences:\n - hidden: Do not show code execution option in preferences (default)\n - disabled: Show selector, but default to no code execution (Vega-Lite spec only)\n - prompt: Show selector, default to prompting user for permission to execute\n - llm: Show selector, default to LLM-validated code execution\n - allow: Show selector, default to executing code without confirmation\n\n WARNING: The 'prompt', 'llm', and 'allow' modes execute LLM-generated code and\n must NEVER be enabled in production environments with access to secrets, credentials,\n or sensitive data. The LLM validation layer reduces accidental errors but does NOT\n provide meaningful security against malicious or compromised inputs.\n ")
class-attribute
instance-attribute
¶
title = param.String(default='Lumen AI - Data Explorer', doc='Title of the app.')
class-attribute
instance-attribute
¶
FileSourceControls
¶
Bases:
Intermediate base class for controls that process files (upload/download).
Adds file-card machinery on top of BaseSourceControls lifecycle
management. UploadSourceControls and DownloadSourceControls
inherit from this class.
add = param.Event(doc='Use uploaded file(s)')
class-attribute
instance-attribute
¶
add_button_icon = param.String(default='add', doc='\n Material icon name for the add/upload confirmation button.')
class-attribute
instance-attribute
¶
add_button_label = param.String(default='Confirm file(s)', doc='\n Text label for the add/upload confirmation button.')
class-attribute
instance-attribute
¶
clear_uploads = param.Boolean(default=True, doc='Clear uploaded file tabs')
class-attribute
instance-attribute
¶
filedropper_kwargs = param.Dict(default={}, doc="Keyword arguments to pass to FileDropper.\n Common options include 'accepted_filetypes' and 'max_file_size'.\n See https://panel.holoviz.org/reference/widgets/FileDropper.html for all options.")
class-attribute
instance-attribute
¶
multiple = param.Boolean(default=True, doc='Allow multiple files')
class-attribute
instance-attribute
¶
replace_controls = param.Boolean(default=False, doc='Replace controls on add')
class-attribute
instance-attribute
¶
upload_handlers = param.Dict(default={}, doc='Handlers for custom file extensions')
class-attribute
instance-attribute
¶
LLMConfigDialog
¶
Bases:
Dialog component for LLM configuration.
Provides: - Provider selection - Temperature slider - Model selection for different model types - Apply/Cancel buttons
agent_types = param.List(default=[], doc='Available agents in the UI')
class-attribute
instance-attribute
¶
llm = param.ClassSelector(class_=Llm, doc='The LLM instance to configure')
class-attribute
instance-attribute
¶
llm_choices = param.List(default=[], doc='Available model choices to display in dropdowns')
class-attribute
instance-attribute
¶
on_llm_change = param.Callable(default=None, doc='Callback function called when LLM provider changes')
class-attribute
instance-attribute
¶
open
property
writable
¶
Get dialog open state.
provider_choices = param.Dict(default={}, doc='Available LLM providers to choose from')
class-attribute
instance-attribute
¶
close()
¶
Close the dialog.
Llm
¶
Bases:
Base class for LLM implementations with standardized client caching.
Subclasses MUST implement _create_base_client(). Subclasses MAY set _instructor_wrapper (default: "openai"). Subclasses MAY override _create_instructor_client() or _get_completion_method().
create_kwargs = param.Dict(default={'max_retries': 1}, doc='\n Additional keyword arguments to pass to the LLM provider\n when calling chat.completions.create. Defaults to no instructor retries\n since agents handle retries themselves.')
class-attribute
instance-attribute
¶
display_name = param.String(default='', constant=True, doc='Display name for UI')
class-attribute
instance-attribute
¶
interceptor = param.ClassSelector(default=None, class_=Interceptor, doc='\n Intercepter instance to intercept LLM calls, e.g. for logging.')
class-attribute
instance-attribute
¶
logfire_tags = param.List(default=None, doc='\n Whether to log LLM calls and responses to logfire.\n If a list of tags is provided, those tags will be used for logging.\n Suppresses streaming responses if enabled since\n logfire does not track token usage on stream.')
class-attribute
instance-attribute
¶
mode = param.Selector(default=(Mode.JSON_SCHEMA), objects=BASE_MODES, doc='\n The calling mode used by instructor to guide the LLM towards generating\n outputs matching the schema.')
class-attribute
instance-attribute
¶
model_kwargs = param.Dict(default={}, doc='\n LLM model definitions indexed by type. Supported types include\n \'default\', \'reasoning\' and \'sql\'. Agents may pick which model to\n invoke for different reasons.\n\n An entry may declare an optional \'description\' (shown to the routing\n model so it can reason about what each option is for) and an optional\n \'routing\' key to opt in to model routing for that type: the routing\n model is invoked first to pick which entry to actually use for each\n call, e.g.\n {"default": {"model": "gpt-5.4-mini"},\n "edit": {"model": "gpt-5.2",\n "description": "Best for editing tables and visualizations",\n "routing": {"model": "nemotron-switchyard"}}}')
class-attribute
instance-attribute
¶
select_models = param.List(default=[], constant=True, doc='Available models for selection dropdowns')
class-attribute
instance-attribute
¶
spec_descriptions = param.Dict(default=SPEC_DESCRIPTIONS, doc='\n Mapping of spec key to human-readable description, used as a\n fallback in the routing prompt when ``model_kwargs`` entries\n do not declare their own ``description``. Seeded with the spec\n keys every provider shares and enriched by the UI layer from\n agent class metadata.')
class-attribute
instance-attribute
¶
temperature = param.Number(default=0.7, bounds=(0, None), allow_None=True, constant=True, doc='\n The temperature to use for sampling. Set to None to omit the\n temperature from requests entirely, e.g. for models that reject it.')
class-attribute
instance-attribute
¶
timeout = param.Number(default=120, bounds=(1, None), constant=True, doc='\n The timeout in seconds for API calls.')
class-attribute
instance-attribute
¶
tools = param.List(default=[], doc='\n Default tools that are always available to this LLM instance.\n These are combined with any tools passed directly to invoke()\n or stream() calls. Accepts the same types as the tools argument\n on those methods (dicts, FunctionTool, or MCPTool instances).')
class-attribute
instance-attribute
¶
usage = UsageCollector()
instance-attribute
¶
usage_pricing = param.Dict(default={}, doc='USD per million tokens by model:\n {"model-name": {"input": 1.0, "cached": 0.1, "cache_write": 1.25,\n "output": 4.0}}.\n Unknown models retain token counts but have no estimated cost.')
class-attribute
instance-attribute
¶
capture_usage()
¶
Capture provider usage for this LLM in the current async context.
initialize(log_level)
async
¶
invoke(messages, system='', response_model=None, allow_partial=False, model_spec='default', tools=None, submit_tool=None, **input_kwargs)
async
¶
Invokes the LLM and returns its response.
Arguments
messages: list[Message]
A list of messages to feed to the LLM.
system: str
A system message to provide to the LLM.
response_model: BaseModel | None
A Pydantic model that the LLM should materialize.
allow_partial: bool
Whether to allow the LLM to only partially fill
the provided response_model.
tools: list[dict[str, Any] | FunctionTool | MCPTool] | None
Tool definitions, FunctionTool, or MCPTool instances to pass through.
When response_model is also given, the client runs tool calls in a loop
until none are requested, then requests the structured response (tools may be unused).
Tool errors, unknown tools and invalid arguments are returned to the model as
tool results; max_tool_rounds (default 16) caps the rounds.
submit_tool: SubmitTool | None
Offer response_model as a tool so the model can answer inside the tool
loop. Requires response_model.
model: Literal['default' | 'reasoning' | 'sql']
The model as listed in the model_kwargs parameter
to invoke to answer the query.
Returns:
| Type | Description |
|---|---|
The completed response_model.
|
|
models()
¶
Return the set of available model identifiers from this provider.
By default returns an empty set. Subclasses should override to query their provider's API for available models.
Returns:
| Type | Description |
|---|---|
|
Set of available model identifiers. |
run_client(model_spec, messages, **kwargs)
async
¶
stream(messages, system='', response_model=None, field=None, model_spec='default', tools=None, **kwargs)
async
¶
Invokes the LLM and streams its response.
Arguments
messages: list[Message] A list of messages to feed to the LLM. system: str A system message to provide to the LLM. response_model: type[BaseModel] | None A Pydantic model that the LLM should materialize. field: str The field in the response_model to stream. tools: list[dict[str, Any] | FunctionTool | MCPTool] | None Tool definitions, FunctionTool, or MCPTool instances to pass through. model: Literal['default' | 'reasoning' | 'sql'] The model as listed in the model_kwargs parameter to invoke to answer the query.
Yields:
| Type | Description |
|---|---|
The string or response_model field.
|
|
trace()
¶
Capture model round trips and tool calls in the current async context.
warmup(model_kwargs)
classmethod
¶
Allows LLM provider to perform actions that ensure that the model(s) are ready to run, e.g. downloading the model files.
LumenEditor
¶
Bases:
component = param.ClassSelector(class_=Component)
class-attribute
instance-attribute
¶
editor = self._render_editor()
instance-attribute
¶
export_formats = ('yaml',)
class-attribute
instance-attribute
¶
footer = param.List(default=[])
class-attribute
instance-attribute
¶
language = 'yaml'
class-attribute
instance-attribute
¶
loading = param.Boolean()
class-attribute
instance-attribute
¶
spec = param.String(allow_None=True)
class-attribute
instance-attribute
¶
title = param.String(allow_None=True)
class-attribute
instance-attribute
¶
view = ParamMethod(self.render, inplace=True, sizing_mode='stretch_width')
instance-attribute
¶
export(fmt)
¶
render()
async
¶
render_context()
async
¶
render_controls(task, interface)
¶
validate_spec(spec)
classmethod
¶
OpenAI
¶
Bases: ,
An LLM implementation using the OpenAI cloud.
api = param.Selector(default='chat_completions', objects=['chat_completions', 'responses'], doc='\n OpenAI API primitive to use.\n - ``chat_completions``: Uses ``/v1/chat/completions`` (default)\n - ``responses``: Uses ``/v1/responses``')
class-attribute
instance-attribute
¶
display_name = param.String(default='OpenAI', constant=True)
class-attribute
instance-attribute
¶
mode = param.Selector(default=(Mode.TOOLS))
class-attribute
instance-attribute
¶
model_kwargs = param.Dict(default={'default': {'model': 'gpt-6-luna'}})
class-attribute
instance-attribute
¶
select_models = param.List(default=['gpt-6-luna', 'gpt-5.6-luna', 'gpt-6-sol', 'gpt-5.4-nano'], constant=True, doc='Warning: Reasoning models (gpt-5, o4-mini) are much slower and not suitable for dialog interfaces.')
class-attribute
instance-attribute
¶
temperature = param.Number(default=0.25, bounds=(0, None), allow_None=True, constant=True)
class-attribute
instance-attribute
¶
get_client(model_spec, response_model=None, **kwargs)
async
¶
models()
¶
Return the set of available model identifiers from OpenAI.
run_client(model_spec, messages, **kwargs)
async
¶
Pipeline
¶
Bases: ,
Pipeline encapsulates filters and transformations applied to a
:class:lumen.sources.base.Source table.
A Pipeline ingests data from a
:class:lumen.sources.base.Source table or another Pipeline
applying the declared :class:lumen.filters.base.Filter,
:class:lumen.transforms.base.Transform and
:class:lumen.transforms.sql.SQLTransform definitions. It can be
used to drive one or more visual outputs or leveraged as a
standalone component to encapsulate multiple data processing
steps.
auto_update = param.Boolean(default=True, constant=True, doc='\n Whether changes in filters, transforms and references automatically\n trigger updates in the data or whether an update has to be triggered\n manually using the update event or the update button in the UI.')
class-attribute
instance-attribute
¶
control_panel
property
¶
data = DataFrame(doc='The current data on this source.')
class-attribute
instance-attribute
¶
dataframe_backend = param.Selector(default=None, objects=[None, *DATAFRAME_BACKENDS], doc='\n The dataframe library `Pipeline.data` is returned as. Leave unset to\n get whatever the Source produced, which avoids a conversion.')
class-attribute
instance-attribute
¶
filters = param.List(item_type=Filter, doc='\n A list of Filters to apply to the source data.')
class-attribute
instance-attribute
¶
pipeline = param.ClassSelector(class_=None, doc='\n Optionally a pipeline may be chained to another pipeline.')
class-attribute
instance-attribute
¶
refs
property
¶
schema = param.Dict(doc='The schema of the input data.')
class-attribute
instance-attribute
¶
source = param.ClassSelector(class_=Source, doc='\n The Source this pipeline is fed by.')
class-attribute
instance-attribute
¶
sql_pushdown = param.Boolean(default=True, doc='\n Whether to narrow the query to the columns the transforms actually\n need when the Source supports SQL, instead of fetching every column\n and projecting in pandas. Only applied where the result is provably\n identical; set to False to always fetch the full table.')
class-attribute
instance-attribute
¶
sql_transforms = param.List(item_type=SQLTransform, doc='\n A list of SQLTransforms to apply to the source data.')
class-attribute
instance-attribute
¶
table = param.String(doc='\n The name of the table driving this pipeline.')
class-attribute
instance-attribute
¶
transforms = param.List(item_type=Transform, doc='\n A list of Transforms to apply to the source data.')
class-attribute
instance-attribute
¶
update = param.Event(label='Apply update', doc='\n Update event trigger (if manual update is set).')
class-attribute
instance-attribute
¶
add_filter(filt, field=None, **kwargs)
¶
Add a filter to the pipeline.
Arguments
filt: Filter | Type[Filter] The filter instance or filter type to add. field: str | None The field to filter on (required to instantiate Filter type).
add_transform(transform, **kwargs)
¶
Add a (SQL)Transform to the pipeline.
Arguments
filt: Transform The Transform instance to add.
chain(filters=None, transforms=None, sql_transforms=None, **kwargs)
¶
Chains additional filtering, transform and sql_transform operations on an existing pipeline. Note that if one or more sql_transforms are provided the existing table will be mirrored into a DuckDB database.
Arguments
filters: List[Filter] | None Additional filters to apply on top of existing pipeline. transforms: List[Transform] | None Additional transforms to apply on top of existing pipeline. sql_transforms: List[SQLTransform] | None Additional filters to apply on top of existing pipeline.
Returns:
| Type | Description |
|---|---|
|
|
clone(**params)
¶
Create a new instance of the pipeline with optionally overridden parameter values.
from_spec(spec, source=None, source_filters=None)
classmethod
¶
Creates a Pipeline from a specification.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
spec
|
|
Specification declared as a dictionary of parameter values or a string referencing a source in the sources dictionary. |
required |
Returns:
| Type | Description |
|---|---|
Resolved and instantiated Pipeline object
|
|
get_dataset()
¶
Return the pipeline data as a gridded xarray.Dataset when the
source can produce one (xarray-backed, via to_dataset) and this is a
plain source query -- no chained pipeline, ParamFilter, or Python
transforms. Returns None otherwise so callers fall back to the
long-form frame.
Lets gridded views render from a compact xarray grid instead of pivoting the long-form data back to xarray.
get_schema()
¶
Generates a JSON schema for the current data held by the Pipeline.
Returns:
| Name | Type | Description |
|---|---|---|
schema |
|
JSON schema for each column in the current data. |
precache(queries)
¶
Populates the cache of the :class:lumen.sources.base.Source with the provided queries.
Queries can be provided in two formats:
-
A dictionary containing 'filters' and 'variables' dictionaries each containing lists of values to compute a cross-product for, e.g.
{ 'filters': {
': ['a', 'b', 'c', ...], ... }, 'variables': { : [0, 2, 4, ...], ... } } - A list containing dictionaries of explicit values for each filter and variables. [{ 'filters': {
: 'a'}, 'variables': { : 0} }, { 'filters': { : 'a'}, 'variables': { : 1} }, ... ]
to_spec(context=None)
¶
Exports the full specification to reconstruct this component.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
context
|
|
Context contains the specification of all previously serialized components, e.g. to allow resolving of references. |
None
|
Returns:
| Type | Description |
|---|---|
Declarative specification of this component.
|
|
traverse(type)
¶
Returns all Filter or Transform objects in a potentially chained pipeline.
validate(spec, context=None)
classmethod
¶
Plan
¶
Bases:
A Plan is a Task that is a collection of other Tasks.
abort_on_error = param.Boolean(default=True, doc='\n If True, the report will abort if an error occurs.')
class-attribute
instance-attribute
¶
agents = param.List(item_type=Actor, default=[])
class-attribute
instance-attribute
¶
coordinator = param.ClassSelector(class_=(param.Parameterized))
class-attribute
instance-attribute
¶
interface = param.ClassSelector(class_=ChatFeed)
class-attribute
instance-attribute
¶
is_followup = param.Boolean(default=False)
class-attribute
instance-attribute
¶
execute(context=None, **kwargs)
async
¶
render_task_history(i=None, failed=False)
¶
Planner
¶
Bases:
The Planner develops a plan to solve the user query step-by-step and then executes it.
planner_tools = param.List(default=[MetadataLookup], doc='\n List of tools to use to provide context for the planner prior\n to making a plan.')
class-attribute
instance-attribute
¶
prompts = param.Dict(default={'main': {'template': PROMPTS_DIR / 'Planner' / 'main.jinja2', 'response_model': make_plan_model}, 'follow_up': {'template': PROMPTS_DIR / 'Planner' / 'follow_up.jinja2', 'response_model': FollowUpClassification}, 'clarification_check': {'template': PROMPTS_DIR / 'Planner' / 'clarification_check.jinja2', 'response_model': ThinkingYesNo}})
class-attribute
instance-attribute
¶
sync(context)
async
¶
Sync both main tools and planner tools.
SQLAgent
¶
Bases:
clean_data = param.Boolean(default=True, doc='\n Profile a bounded sample of each query result and, when it contains data-quality problems,\n spend one extra LLM call rewriting the query to clean them up.\n Profiling is deterministic and reuses the frame validation already\n fetched, so a clean result costs nothing; set to False to always take\n the query exactly as first written.')
class-attribute
instance-attribute
¶
conditions = param.List(default=['Use for querying, filtering, aggregating, or transforming data with SQL', "Use for calculations that require executing SQL (e.g., 'calculate average', 'sum by category')", "Use when user asks to 'show', 'get', 'fetch', 'query', 'find', 'filter', 'calculate', 'aggregate', or 'transform' data", 'Use after external data has been fetched and the user expects the fetched fields/rows to be presented, selected, filtered, or otherwise queried', 'Use when user asks about data quality, completeness, missing values, duplicates, or outliers', "NOT when user asks to 'explain', 'interpret', 'analyze', 'summarize', or 'comment on' existing data", 'NOT useful if the user is using the same data for plotting'])
class-attribute
instance-attribute
¶
exclusions = param.List(default=['dbtsl_metaset'])
class-attribute
instance-attribute
¶
input_schema = SQLInputs
class-attribute
instance-attribute
¶
llm_tools = param.List(default=[make_apply_filter_llm_tool])
class-attribute
instance-attribute
¶
max_request_llm_calls = param.Integer(default=40, bounds=(1, None), doc='\n Maximum LLM calls one SQL request may spend across all attempts, tool\n rounds and revisions. Checked before each attempt.')
class-attribute
instance-attribute
¶
not_with = param.List(default=['DbtslAgent', 'MetadataLookup', 'TableListAgent'])
class-attribute
instance-attribute
¶
output_schema = SQLOutputs
class-attribute
instance-attribute
¶
prompts = param.Dict(default={'main': {'response_model': make_sql_model, 'template': PROMPTS_DIR / 'SQLAgent' / 'main.jinja2'}, 'clean_data': {'response_model': SQLCleanup, 'template': PROMPTS_DIR / 'SQLAgent' / 'clean_data.jinja2'}, 'revise_output': {'response_model': RetrySpec, 'template': PROMPTS_DIR / 'SQLAgent' / 'revise_output.jinja2'}})
class-attribute
instance-attribute
¶
purpose = param.String(default='\n Creates and executes SQL queries to retrieve, filter, aggregate, or transform data.\n Handles table joins, WHERE clauses, GROUP BY, calculations, and other SQL operations.\n Generates new data pipelines from SQL transformations.\n Profiles every result for data-quality problems (missing values, duplicates,\n placeholder numbers, outliers) and reports them.')
class-attribute
instance-attribute
¶
query_timeout = param.Number(default=60, bounds=(0, None), allow_None=True, doc="\n Seconds a query run on the model's behalf may take before it is\n abandoned, or None to wait indefinitely. A timed out query is\n revised once; a second timeout ends the request.")
class-attribute
instance-attribute
¶
request_timeout = param.Number(default=300, bounds=(0, None), allow_None=True, doc='\n Wall-clock limit in seconds for one SQL request across all attempts.\n None disables the limit.')
class-attribute
instance-attribute
¶
schema_max_tokens = param.Integer(default=4000, allow_None=True, doc='\n Token budget for the data summary in the prompt. Tables beyond it are\n listed with names and types only, and the model loads the rest with\n load_table_schemas.')
class-attribute
instance-attribute
¶
schema_tables_shown = param.Integer(default=25, bounds=(1, None), doc='\n Most relevant tables described in the data summary; the rest are\n listed by name. schema_max_tokens usually binds first.')
class-attribute
instance-attribute
¶
stats_wait = param.Number(default=10, allow_None=True, doc='\n Seconds to wait for column statistics that are still being computed\n before answering with names and types; profiling continues in the\n background for later questions. None waits until they are ready.')
class-attribute
instance-attribute
¶
user = param.String(default='SQL')
class-attribute
instance-attribute
¶
respond(messages, context, step_title=None)
async
¶
Generate SQL in one step (catalog browse, schema load, optional exploration, then structured query).
revise(instruction, messages, context, view=None, spec=None, language=None, errors=None, max_retries=2, **kwargs)
async
¶
SQLEditor
¶
Section
¶
Bases:
A Section is a TaskGroup representing a sequence of related tasks.
level = 2
class-attribute
instance-attribute
¶
SourceAgent
¶
Bases:
Fetches data from external APIs, Python callables, or URL templates configured via source controls.
The agent reads available actions from SourceLookup context,
wraps the relevant callables as FunctionTool instances, and
passes them to the LLM. The LLM decides which action to call and
fills in parameter values. Results are normalized to DataFrames
and registered as DuckDBSource entries for downstream agents
(e.g. SQLAgent, hvPlotAgent) to consume.
conditions = param.List(default=['Use when no existing data source can answer the query and external source controls are configured', 'Use when the user asks for data from an external API or service', 'Use when the user provides a URL to read, fetch, discuss, or extract content from — the content must be fetched before any other agent can address it', "Use when the loaded data does not cover the requested scope (e.g. user asks for 'whole month' but data summary only shows a week)", 'Prefer over metadata discovery when no data sources are loaded but source controls exist', 'This agent fetches and registers data or indexes page content as documents — always follow with a presentation, query, or chat step', 'For HTML pages, this agent extracts both tabular data and page text (indexed as a searchable document for ChatAgent)', 'Output is queryable tabular data and/or indexed documents — enables filtering, aggregation, joins, transformation, and conversation by downstream agents', "NOT for listing configured external source actions or answering 'what data is available' before a concrete dataset request", 'NOT when the loaded data already covers the requested scope and user just wants to query/visualize it'])
class-attribute
instance-attribute
¶
input_schema = SourceInputs
class-attribute
instance-attribute
¶
output_schema = SourceOutputs
class-attribute
instance-attribute
¶
prompts = param.Dict(default={'main': {'template': PROMPTS_DIR / 'SourceAgent' / 'main.jinja2'}})
class-attribute
instance-attribute
¶
purpose = param.String(default='\n Fetches data from configured external sources (APIs, Python callables,\n URL templates). Discovers relevant endpoints, fills parameters via\n LLM, executes the call, and registers the resulting DataFrame for\n downstream querying and visualization. For HTML pages, also indexes\n page text as a searchable document for conversational agents.')
class-attribute
instance-attribute
¶
user = param.String(default='Source')
class-attribute
instance-attribute
¶
applies(context)
async
classmethod
¶
respond(messages, context, step_title=None)
async
¶
SourceCatalog
¶
Bases:
A component that displays all data sources and documents as hierarchical trees.
context = param.Dict(default={})
class-attribute
instance-attribute
¶
sources = param.List(default=[], doc='\n List of data sources to display in the catalog.')
class-attribute
instance-attribute
¶
vector_store = param.Parameter(default=None, doc='\n The vector store to sync metadata documents to. If not provided,\n metadata files will only be stored locally.')
class-attribute
instance-attribute
¶
visibility_changed = param.Event(doc='\n Triggered when table or document visibility changes.')
class-attribute
instance-attribute
¶
sync(context=None)
¶
StoryReport
¶
Bases:
A Report with an extra Story tab: an AI writes a blog-post story that
interleaves prose with the report's charts and tables, which can then be
edited by hand or with the LLM, regenerated, and kept as versions.
reset(start=0)
¶
TableExplorer
¶
Bases:
TableExplorer provides a high-level entrypoint to explore tables in a split UI. It allows users to load tables, explore them using Graphic Walker, and then interrogate the data via a chat interface.
add_exploration = param.Event(label='Explore table')
class-attribute
instance-attribute
¶
context = param.Dict(default={})
class-attribute
instance-attribute
¶
source_map = {}
instance-attribute
¶
table_slug = param.Selector(label='Select table(s) to preview.')
class-attribute
instance-attribute
¶
create_sql_output()
¶
sync(context=None)
async
¶
TableListAgent
¶
Bases:
The TableListAgent lists all available data and lets the user pick one.
conditions = param.List(default=["Use when user explicitly asks to 'list data', 'show available data', or 'what data do you have'", 'NOT for showing actual data contents, querying, or analyzing data', 'NOT for describing data sources or telling what the data is about'])
class-attribute
instance-attribute
¶
input_schema = TableListInputs
class-attribute
instance-attribute
¶
not_with = param.List(default=['DbtslAgent', 'SQLAgent'])
class-attribute
instance-attribute
¶
purpose = param.String(default='\n Displays a list of all available data & datasets. Not useful for identifying which dataset to use for analysis.')
class-attribute
instance-attribute
¶
user = param.String(default='Tables')
class-attribute
instance-attribute
¶
applies(context)
async
classmethod
¶
UI
¶
Bases:
UI provides a baseclass and high-level entrypoint to start chatting with your data.
agents = param.List(default=[], doc='\n List of additional Agents to add beyond the default_agents.')
class-attribute
instance-attribute
¶
analyses = param.List(default=[], doc='\n List of custom analyses. If provided the AnalysesAgent will be added.')
class-attribute
instance-attribute
¶
context = param.Dict(default={})
class-attribute
instance-attribute
¶
coordinator = param.ClassSelector(class_=Coordinator, default=Planner, is_instance=False, doc='\n The Coordinator class that will be responsible for coordinating the Agents.')
class-attribute
instance-attribute
¶
coordinator_params = param.Dict(default={}, doc='Keyword arguments to pass to the Coordinator.')
class-attribute
instance-attribute
¶
default_agents = param.List(default=[TableListAgent, ChatAgent, DocumentListAgent, DocumentSummarizerAgent, SQLAgent, SourceAgent, VegaLiteAgent, ValidationAgent, DeckGLAgent], doc='List of default agents which will always be added.')
class-attribute
instance-attribute
¶
demo_inputs = param.List(default=DEMO_MESSAGES, doc='\n List of instructions to demo the Coordinator.')
class-attribute
instance-attribute
¶
document_vector_store = param.ClassSelector(class_=VectorStore, default=None, doc='\n The vector store to use for document-related tools. If not provided, a new one will be created\n or inferred from the tools provided.')
class-attribute
instance-attribute
¶
export_functions = param.Dict(default={}, doc='\n Dictionary mapping from name of exporter to export function.')
class-attribute
instance-attribute
¶
filedropper_kwargs = param.Dict(default={}, doc="Keyword arguments to pass to FileDropper in UploadSourceControls.\n Common options include 'accepted_filetypes' and 'max_file_size'.\n See https://panel.holoviz.org/reference/widgets/FileDropper.html for all available options.")
class-attribute
instance-attribute
¶
interface = param.ClassSelector(class_=ChatFeed, doc='\n The interface for the Coordinator to interact with.')
class-attribute
instance-attribute
¶
llm = param.ClassSelector(class_=Llm, default=None, doc='\n The LLM provider to use. Detected from the environment when the UI\n is created if not provided.')
class-attribute
instance-attribute
¶
llm_choices = param.List(default=[], doc='\n List of available LLM model choices to show in the configuration dialog.')
class-attribute
instance-attribute
¶
llm_tools = param.List(default=[], doc="\n Extra tools for the coordinator's :attr:`~lumen.ai.actor.LLMUser.llm_tools`\n (expanded in ``_invoke_prompt`` / ``_stream_prompt`` only).")
class-attribute
instance-attribute
¶
log_level = param.ObjectSelector(default='DEBUG', objects=['DEBUG', 'INFO', 'WARNING', 'ERROR'], doc='\n The log level to use.')
class-attribute
instance-attribute
¶
logfire_tags = param.List(default=None, doc='\n Whether to log LLM calls and responses to logfire.\n If a list of tags is provided, those tags will be used for logging.\n Suppresses streaming responses if enabled since\n logfire does not track token usage on stream.')
class-attribute
instance-attribute
¶
logs_db_path = param.String(default=None, constant=True, doc='\n The path to the log file that will store the messages exchanged with the LLM.')
class-attribute
instance-attribute
¶
notebook_preamble = param.String(default='', doc='\n Preamble to add to exported notebook(s).')
class-attribute
instance-attribute
¶
page_config = param.Dict(default={}, doc='\n Configuration for the panel-material-ui Page component the UI is rendered into.')
class-attribute
instance-attribute
¶
provider_choices = param.Dict(default={}, doc='\n Available LLM providers to show in the configuration dialog.')
class-attribute
instance-attribute
¶
source_controls = param.List(default=[UploadSourceControls, DownloadSourceControls], doc='\n List of SourceControls types to manage datasets.')
class-attribute
instance-attribute
¶
suggestions = param.List(default=GETTING_STARTED_SUGGESTIONS, doc='\n Initial list of suggestions of actions the user can take.')
class-attribute
instance-attribute
¶
title = param.String(default='Lumen UI', doc='Title of the app.')
class-attribute
instance-attribute
¶
tools = param.List(doc='\n List of Tools that can be invoked by the coordinator.')
class-attribute
instance-attribute
¶
upload_handlers = param.Dict(default={}, doc='\n Dictionary mapping from file extensions to handler function,\n e.g. {"hdf5": ...}. The callback function should accept the file bytes,\n table alias, and filename, add or modify the `sources` in memory, and return a bool\n (True if the table was successfully uploaded).')
class-attribute
instance-attribute
¶
vector_store = param.ClassSelector(class_=VectorStore, default=None, doc='\n The vector store to use for tools unrelated to documents. If not provided, a new one will be created\n or inferred from the tools provided.')
class-attribute
instance-attribute
¶
servable(title=None, **kwargs)
¶
show(**kwargs)
¶
UploadSourceControls
¶
Bases:
Controls for uploading files from the local filesystem.
add_button_icon = 'upload_file'
class-attribute
instance-attribute
¶
add_button_label = 'Upload file(s)'
class-attribute
instance-attribute
¶
label = '<span class="material-icons" style="vertical-align: middle;">upload</span> Upload Data'
class-attribute
instance-attribute
¶
load_mode = 'manual'
class-attribute
instance-attribute
¶
ValidationAgent
¶
Bases:
ValidationAgent focuses solely on validating whether the executed plan fully answered the user's original query. It identifies missing elements and suggests next steps when validation fails.
conditions = param.List(default=['Use to validate whether executed plans fully answered user queries', 'Use to identify missing elements from the original user request', 'NOT for data analysis, pattern identification, or technical programming questions'])
class-attribute
instance-attribute
¶
input_schema = ValidationInputs
class-attribute
instance-attribute
¶
output_schema = ValidationOutputs
class-attribute
instance-attribute
¶
prompts = param.Dict(default={'main': {'template': PROMPTS_DIR / 'ValidationAgent' / 'main.jinja2', 'response_model': QueryCompletionValidation, 'tools': []}})
class-attribute
instance-attribute
¶
purpose = param.String(default="\n Validates whether executed plans fully answered the user's original query.\n Identifies missing elements, assesses completeness, and suggests next steps\n when validation fails. Acts as a quality gate for plan execution.")
class-attribute
instance-attribute
¶
user = param.String(default='Validation')
class-attribute
instance-attribute
¶
respond(messages, context, step_title=None)
async
¶
VectorStore
¶
Bases:
Abstract base class for a vector store.
chunk_func = param.Callable(default=None, doc='\n The function used to split documents into chunks.\n Must accept `text`. If None, defaults to semchunk.chunkerify\n with the chunk_size.')
class-attribute
instance-attribute
¶
chunk_func_kwargs = param.Dict(default={}, doc='\n Additional keyword arguments to pass to the chunking function.')
class-attribute
instance-attribute
¶
chunk_size = param.Integer(default=512, doc='Maximum size of text chunks to split documents into.')
class-attribute
instance-attribute
¶
chunk_tokenizer = param.String(default='gpt-4o-mini', doc='\n If using the default chunk_func, the tokenizer used to split documents into chunks.\n Must be a valid tokenizer name from the transformers or tiktoken library.\n Otherwise, please pass in a custom chunk_func.')
class-attribute
instance-attribute
¶
embeddings = param.ClassSelector(class_=Embeddings, default=(NumpyEmbeddings()), allow_None=True, doc='Embeddings object for text processing.')
class-attribute
instance-attribute
¶
excluded_metadata = param.List(default=['llm_context'], doc='List of metadata keys to exclude when creating the embeddings.')
class-attribute
instance-attribute
¶
max_concurrent = param.Integer(default=5, bounds=(1, None), doc="\n Maximum number of files to process concurrently in add_directory.\n Set to 1 for backends that don't support concurrent writes.")
class-attribute
instance-attribute
¶
prompts = param.Dict(default={'main': {'template': PROMPTS_DIR / 'VectorStore' / 'main.jinja2'}, 'should_situate': {'template': PROMPTS_DIR / 'VectorStore' / 'should_situate.jinja2', 'response_model': YesNo}}, doc="\n A dictionary of prompts used by the vector store, indexed by prompt name.\n Each prompt should be defined as a dictionary containing a template\n 'template' and optionally a 'model' and 'tools'.")
class-attribute
instance-attribute
¶
situate = param.Boolean(default=False, doc='\n Whether to insert a `llm_context` key in the metadata containing\n contextual about the chunks.')
class-attribute
instance-attribute
¶
add(items, force_ids=None, situate=None)
async
¶
Add items to the vector store.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
items
|
|
List of dictionaries containing 'text' and optional 'metadata'. |
required |
force_ids
|
|
Optional list of IDs to use instead of generating new ones. |
None
|
situate
|
|
Whether to insert a |
None
|
Returns:
| Type | Description |
|---|---|
List of assigned IDs for the added items.
|
|
add_directory(directory, pattern='*', exclude_patterns=None, metadata=None, situate=None, upsert=False, raise_on_error=False, max_concurrent=None)
async
¶
Recursively add files from a directory that match the pattern and don't match exclude patterns.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
directory
|
|
The path to the directory to search for files. |
required |
pattern
|
|
Glob pattern to match files against (e.g., ".txt", ".py"). Default is "*". |
'*'
|
exclude_patterns
|
|
List of patterns to exclude. Files matching any of these patterns will be skipped. |
None
|
metadata
|
|
Base metadata to apply to all files. Will be extended with filename-specific metadata. |
None
|
situate
|
|
Whether to insert a |
None
|
upsert
|
|
If True, will update existing items if similar content is found. Default is False. |
False
|
raise_on_error
|
If True, will raise an error if any file fails to process. Default is False. |
False
|
|
max_concurrent
|
|
Maximum number of files to process concurrently. If None, uses self.max_concurrent. |
None
|
Returns:
| Type | Description |
|---|---|
|
Combined list of IDs for all added files. |
add_file(filename, ext=None, metadata=None, situate=None, upsert=False)
async
¶
Adds a file or a URL to the collection.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
filename
|
|
The path to the file, a file-like object or a URL to be added. |
required |
ext
|
|
The file extension to associate with the added file. If not provided, it will be determined from the file or URL. |
None
|
metadata
|
|
A dictionary containing metadata related to the file (e.g., title, author, description). Defaults to None. |
None
|
situate
|
|
Whether to insert a |
None
|
upsert
|
|
If True, will update existing items if similar content is found. Defaults to False. |
False
|
Returns:
| Type | Description |
|---|---|
List of assigned IDs for the added items.
|
|
clear()
abstractmethod
¶
Clear all items from the vector store.
close()
abstractmethod
¶
Close the vector store and release any resources.
delete(ids)
abstractmethod
¶
Delete items from the vector store by their IDs.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
ids
|
|
List of IDs to delete. |
required |
filter_by(filters, limit=None, offset=0)
abstractmethod
¶
Filter items by metadata without using embeddings similarity.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
filters
|
|
Dictionary of metadata key-value pairs to filter by. |
required |
limit
|
|
Maximum number of results to return. If None, returns all matches. |
None
|
offset
|
|
Number of results to skip (for pagination). |
0
|
Returns:
| Type | Description |
|---|---|
List of results with 'id', 'text', and 'metadata'.
|
|
query(text, top_k=5, filters=None, threshold=-1.0)
abstractmethod
async
¶
Query the vector store for similar items.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text
|
|
The query text. |
required |
top_k
|
|
Number of top results to return. |
5
|
filters
|
|
Optional metadata filters. |
None
|
threshold
|
|
Minimum similarity score required for a result to be included. |
-1.0
|
Returns:
| Type | Description |
|---|---|
List of results with 'id', 'text', 'metadata', and 'similarity' score.
|
|
should_situate_chunk(chunk)
async
¶
Determine whether a chunk should be situated based on its content.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
chunk
|
|
The chunk text to evaluate |
required |
Returns:
| Type | Description |
|---|---|
|
Whether the chunk should be situated |
upsert(items, situate=None)
async
¶
Add items to the vector store if similar items don't exist, update them if they do.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
items
|
|
List of dictionaries containing 'text' and optional 'metadata'. |
required |
situate
|
|
Whether to insert a |
None
|
Returns:
| Type | Description |
|---|---|
List of assigned IDs for the added or updated items.
|
|
VegaLiteAgent
¶
Bases:
conditions = param.List(default=['Use for publication-ready visualizations or when user specifically requests Vega-Lite charts', 'Use for polished charts intended for presentation or sharing', 'Use for a choropleth whose table names places (states, countries) but holds no geometry or coordinates, since the boundaries can be looked up and joined by name'])
class-attribute
instance-attribute
¶
doc_pages_per_slug = param.Integer(default=2, bounds=(1, None), doc='\n Maximum number of retrieved sections allowed from any single\n documentation page. Without a cap, similarity collapses onto one page:\n a "bar chart" query returns five near-identical mark-bar variants while\n the transform and encoding pages that would broaden the answer sit just\n below the cut.')
class-attribute
instance-attribute
¶
exclude_spec_constructs = param.List(default=['facet', 'repeat', 'concat', 'hconcat', 'vconcat', 'row', 'column'], item_type=str, doc='\n Vega-Lite composition keys that disqualify a retrieved example spec.\n The declarative prompt tells the model not to pack several plots into one\n entry with these, so injecting a reference spec that uses them puts the\n documentation at odds with the instructions. Set to [] to keep every\n retrieved spec -- appropriate for the Altair path, whose prompt permits\n composition within a single chart entry.')
class-attribute
instance-attribute
¶
n_doc_pages = param.Integer(default=5, bounds=(0, None), doc='\n Number of documentation sections to inject into the prompt.\n Set to 0 to disable documentation lookup entirely.')
class-attribute
instance-attribute
¶
prompts = param.Dict(default={'main': {'response_model': VegaLiteSpec, 'template': PROMPTS_DIR / 'VegaLiteAgent' / 'main.jinja2'}, 'main_altair': {'response_model': AltairSpec, 'template': PROMPTS_DIR / 'VegaLiteAgent' / 'main_altair.jinja2'}, 'code_safety': {'response_model': CodeSafetyCheck, 'template': PROMPTS_DIR / 'VegaLiteAgent' / 'code_safety.jinja2'}, 'interaction_polish': {'response_model': VegaLiteSpecUpdate, 'template': PROMPTS_DIR / 'VegaLiteAgent' / 'interaction_polish.jinja2'}, 'annotate_plot': {'response_model': VegaLiteSpecUpdate, 'template': PROMPTS_DIR / 'VegaLiteAgent' / 'annotate_plot.jinja2'}, 'revise_output': {'response_model': RetrySpec, 'template': PROMPTS_DIR / 'VegaLiteAgent' / 'revise_output.jinja2'}})
class-attribute
instance-attribute
¶
purpose = param.String(default='Generates one or more vega-lite plot specifications from the input data pipeline.')
class-attribute
instance-attribute
¶
reserve_base_form = param.Boolean(default=True, doc='\n Guarantee one retrieved section is the base form of a chart type, when\n the candidate pool contains one. A bare request like "plot a bar chart"\n matches every bar variant about equally -- similarity across the top\n candidates spans less than the noise floor -- so the plain section loses\n to specialisations that simply have more text to match on. Sections are\n flagged deterministically at build time from their title, which is why\n this reserves a slot rather than nudging a score.')
class-attribute
instance-attribute
¶
user = param.String(default='Vega')
class-attribute
instance-attribute
¶
vector_store_path = param.Path(default=None, check_exists=False, doc='\n Path to a custom vector store for storing and retrieving Vega-Lite examples;\n if not provided a default store will be used depending on the LLM--\n OpenAIEmbeddings for OpenAI LLM or NumpyEmbeddings for all others.')
class-attribute
instance-attribute
¶
view_type = VegaLiteView
class-attribute
instance-attribute
¶
annotate(instruction, messages, context, spec, view=None)
async
¶
Apply annotations based on user request.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
instruction
|
|
User's description of what to annotate |
required |
messages
|
|
Chat history for context |
required |
context
|
|
Session context |
required |
spec
|
|
The current VegaLite specification (full dict with 'spec' key) |
required |
view
|
|
Optional VegaLite editor to export plot image from for vision analysis. |
None
|
Returns:
| Type | Description |
|---|---|
|
Updated specification with annotations |
respond(messages, context, step_title=None)
async
¶
Generates one or more VegaLite visualizations using a progressive building approach with real-time updates.
revise(feedback, messages, context, view=None, spec=None, language=None, errors=None, **kwargs)
async
¶
Revise a VegaLite specification based on user feedback.
This override adds: 1. Doc examples from vector store 2. Plot image for vision analysis (if view is a VegaLiteEditor)
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
feedback
|
|
User's feedback or instruction for revision. |
required |
messages
|
|
Chat message history. |
required |
context
|
|
Session context. |
required |
view
|
|
The editor containing the current spec to revise. |
None
|
spec
|
|
The spec string (used if view is None). |
None
|
language
|
|
The spec language (used if view is None). |
None
|
errors
|
|
List of errors to include in context. |
None
|
**kwargs
|
Additional arguments passed to parent revise. |
{}
|
Returns:
| Type | Description |
|---|---|
|
The revised YAML specification. |
XArraySQLSource
¶
Bases:
SQL-queryable xarray data source for Lumen.
Wraps xarray datasets with Apache DataFusion (via xarray-sql) to provide full SQL query support. Each data variable in the dataset is exposed as a separate SQL table with coordinate columns.
To use this source, install the optional xarray dependencies::
pip install lumen[xarray]
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
uri
|
|
Path or URL to a NetCDF/Zarr/HDF5/GRIB file. |
required |
engine
|
|
xarray engine to use. Auto-detected from file extension if None. |
required |
chunks
|
|
Chunking specification for dask-backed lazy loading. Use 'auto' for automatic chunking, or a dict like {'time': 100}. |
required |
open_kwargs
|
|
Additional keyword arguments passed to xr.open_dataset(). |
required |
variables
|
|
Subset of data variables to expose. None means all variables. |
required |
Examples:
>>> source = XArraySQLSource(uri="air_temperature.nc")
>>> source.get_tables()
['air']
>>> df = source.execute("SELECT lat, lon, AVG(air) FROM air GROUP BY lat, lon")
>>> schema = source.get_schema("air")
chunks = param.Parameter(default='auto', doc="\n Dask chunk specification for lazy loading.\n Use 'auto', a dict like {'time': 100}, or None for eager loading.")
class-attribute
instance-attribute
¶
dataset
property
¶
Access the underlying xarray Dataset.
engine = param.String(default=None, allow_None=True, doc='\n xarray backend engine. Auto-detected from file extension if None.\n Options: netcdf4, zarr, cfgrib, h5netcdf.')
class-attribute
instance-attribute
¶
open_kwargs = param.Dict(default={}, doc='\n Additional keyword arguments for xr.open_dataset().')
class-attribute
instance-attribute
¶
source_type = 'xarray'
class-attribute
¶
sql_expr = param.String(default='SELECT * FROM {table}', doc='\n The SQL expression template for table queries.')
class-attribute
instance-attribute
¶
tables = param.ClassSelector(class_=(list, dict), default=None, allow_None=True, doc='\n List or dict of tables. Populated automatically from dataset variables.\n When a dict, maps logical table names to SQL expressions.')
class-attribute
instance-attribute
¶
uri = param.String(default=None, allow_None=True, doc='\n Path or URL to the xarray-compatible data file.\n Supports NetCDF (.nc), Zarr (.zarr), HDF5 (.h5), GRIB (.grib).')
class-attribute
instance-attribute
¶
variables = param.List(default=None, allow_None=True, doc='\n Subset of data variables to expose as tables.\n None means all variables are exposed.')
class-attribute
instance-attribute
¶
close()
¶
Clean up resources.
create_sql_expr_source(tables, params=None, **kwargs)
¶
Create a new XArraySQLSource sharing the same DataFusion context.
execute(sql_query, params=None, *args, **kwargs)
¶
Execute a SQL query against the DataFusion context.
fetch(sql_query, params=None)
¶
Fetch query results in dataframe_backend, built by DataFusion.
DataFusion is arrow native, so the to_pandas in execute is a conversion this can skip entirely when another library was asked for.
from_dataset(dataset, chunks='auto', **kwargs)
classmethod
¶
Creates an in-memory XArraySQLSource from an xarray Dataset.
Arguments
dataset: xr.Dataset The xarray Dataset to query. chunks: dict or str Chunk specification for DataFusion registration. kwargs: any Additional keyword arguments for the source.
Returns:
| Name | Type | Description |
|---|---|---|
source |
|
|
get(table, **query)
¶
get_schema(table=None, limit=None, shuffle=False)
¶
get_tables()
¶
normalize_table(table)
¶
to_dataset(table, **query)
¶
Return the query result as a gridded xr.Dataset.
Uses xarray-sql's to_dataset, which is lazy and keeps the data
gridded, so a gridded source is never materialized as long-form.
_keyless_options_message()
¶
Tell a user with no provider configured what works without an API key on this machine.
content_to_text(content)
¶
Extract the text portion from a message content field.
Handles both plain string content and multimodal content lists (e.g. containing both text strings and image objects). Returns just the concatenated text parts.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
content
|
|
The message content — either a plain string or a list of mixed content items (strings, Image objects, etc.). |
required |
Returns:
| Type | Description |
|---|---|
|
The extracted text content. |
export_notebook(outputs, preamble='')
¶
format_msg_content(content)
¶
Format a message content field for logging.
Handles plain strings, multimodal lists (text + images), and
arbitrary objects. Non-string items are represented as compact
Type[id] placeholders so that base64 blobs never flood logs.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
content
|
|
The message content — a string, a list of mixed items, or any object. |
required |
Returns:
| Type | Description |
|---|---|
|
A human-readable, log-safe representation of the content. |
get_available_llm()
¶
Detect an available LLM provider: the first one whose API key environment variable is set.
Providers that need no key, such as Ollama and the coding CLIs, are only used when requested explicitly.
Returns:
| Type | Description |
|---|---|
|
The LLM class, or None if no provider is available. |
local_cli_providers()
¶
Coding CLI providers installed on PATH, mapped to whether they report a signed-in user.
log_debug(msg, offset=24, prefix='', suffix='', show_sep=None, show_length=False)
¶
Log a debug message with a separator line above and below.
normalize_table_name(name)
¶
Normalize a table name to a valid SQL identifier.
Replaces all non-word characters (anything except letters, digits, and underscores) with underscores, strips leading/trailing underscores, and converts to lowercase.
This matches the behavior of DuckDBSource.normalize_table() to ensure consistent table naming across the codebase.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
|
The table name to normalize (e.g., filename without extension) |
required |
Returns:
| Type | Description |
|---|---|
|
The normalized table name |
Examples:
read_geo_file(file_obj, extension, alias)
¶
Parse a geospatial file, returning WKB geometry + conversion SQL.
try_import_xarray(load=True)
¶
Import and return xarray, or None if xarray or xarray-sql is unavailable.
wrap_logfire(span_name=None, extract_args=True, **instrument_kwargs)
¶
Decorator to instrument a function with logfire if llm.logfire_tags is provided. Expects the decorated function to have an 'llm' attribute (on self) or be passed via instrument_kwargs.