Conceptual Overview
The Semantic Model in Lyzr is a system designed to enable intelligent understanding, documentation, and utilization of structured tabular data (such as database tables) in AI workflows. It acts as a bridge between raw relational data and human-readable, AI-consumable knowledge by semantically enriching tables with detailed context.What is a Semantic Model?
A Semantic Model adds meaning and context to structured data by attaching:- Table-level descriptions: High-level summaries that describe what a table represents.
- Column-level metadata: Detailed natural language explanations for each column’s purpose and data type.
- Data previews: Sample rows that provide real-world context to help AI systems and humans better understand usage.
- Retrieval-Augmented Generation (RAG) flows,
- Search and documentation agents, and
- Natural language interfaces over data.
Why Use Semantic Modeling?
Traditional databases lack human-centric explanations, making it hard for:- Developers to understand schemas quickly,
- AI models to answer data-related questions accurately,
- Non-technical users to interact with structured data.
- Generating semantic documentation automatically using LLMs.
- Structuring the output for both human consumption and AI workflows.
- Saving the enhanced information in a vectorized format for fast and relevant retrieval.
Core Workflow
- Input Source: A database table with rows and schema is provided as input.
- LLM-Powered Inference: A language model reviews table structure and sample data to generate descriptions.
- Semantic Description Output:
- What the table is about
- What each column represents
- How the table connects to business or analytical use cases
- Storage & Retrieval:
- These semantic blocks are embedded and stored in a vector database.
- Future retrievals (like question answering or agent planning) can now pull contextually rich, accurate descriptions.
Components of the Semantic Model
| Component | Description |
|---|---|
| Table Name | The identifier of the dataset |
| Table Description | A high-level explanation of the table’s content and intent |
| Columns | A list of column names, each paired with a natural language description and data type |
| Preview Records | Optional rows from the table used for contextual grounding |
| RAG Config | Configuration used to generate or retrieve this semantic documentation |
| Task Handling | For large datasets, semantic documentation can be generated asynchronously |
Benefits
- Improved Discoverability: Semantic metadata makes it easier to search and explore datasets.
- Agent Integration: Documentation agents and RAG models can use this metadata to answer user queries with high precision.
- Auto-Documentation: Automatically generated explanations save time for data engineers and analysts.
- Natural Language Access: Even non-technical users can query data through AI using the semantic layer as a knowledge base.