Dataherald

Introduction

Authentication

Getting Started

Support

Metadata

Database Connection

Add database connection

Retrieve database connection

List all database connections

Update database connection

Golden SQL

Add golden SQL

Retrieve golden SQL

List all golden SQLs

Delete golden SQL

Instruction

Add instruction

Retrieve instruction

List all instructions

Update instruction

Delete instruction

Table Description

Scan tables

Retrieve table description

List all table descriptions

Update table description

Finetuning

Create a finetuning job

Retrieve a finetuning job

List all finetuning jobs

Cancel a finetuning job

Prompt

Create prompt

Retrieve prompt

List all prompts

SQL Generation

Generate SQL

Generate SQL for a Prompt ID

Retrieve SQL generation

List all SQL generations

List all SQL generations for a Prompt ID

Execute SQL

NL Generation

Generate NL Response

Generate SQL+NL Response

Generate Prompt+SQL+NL Response

Retrieve NL generation

List all NL generations

Guides

How to Connect to your Database

How to Ask a Question

How to Add a Golden Record

Improving NL-to-SQL Performance

FastAPI

Dataherald AI

<p class="slate-p" placeholder="Type / for widgets">You can interact with the Dataherald API through HTTP requests from any language or through our official Python SDK package.</p><p class="slate-p" placeholder="Type / for widgets">To install the official Python SDK package, run the following command:</p><p class="slate-p" placeholder="Type / for widgets"> </p><pre class="slate-CodeBlockElement slate-code_block"><div contenteditable="false"><div ><span aria-live="polite" aria-atomic="false" aria-relevant="additions text" ></span><div ><div ><div >Select...</div><div ><div  style="display:inline-block"><input type="text" autoCapitalize="none" autoComplete="off" autoCorrect="off" id="react-select-150-input" spellcheck="false" tabindex="0" value="" aria-autocomplete="list" style="box-sizing:content-box;width:1px;label:input;background:0;border:0;font-size:inherit;opacity:1;outline:0;padding:0;color:inherit"/><div style="position:absolute;top:0;left:0;visibility:hidden;height:0;overflow:scroll;white-space:pre"></div></div></div></div><div ><span ></span><div  aria-hidden="true"><svg height="20" width="20" viewBox="0 0 20 20" aria-hidden="true" focusable="false" ><path d="M4.516 7.548c0.436-0.446 1.043-0.481 1.576 0l3.908 3.747 3.908-3.747c0.533-0.481 1.141-0.446 1.574 0 0.436 0.445 0.408 1.197 0 1.615-0.406 0.418-4.695 4.502-4.695 4.502-0.217 0.223-0.502 0.335-0.787 0.335s-0.57-0.112-0.789-0.335c0 0-4.287-4.084-4.695-4.502s-0.436-1.17 0-1.615z"></path></svg></div></div></div></div></div><code class="language-python keep-markup drop-tokens"><div class="slate-code_line">pip install dataherald</div></code></pre><p class="slate-p" placeholder="Type / for widgets">You can join our <a href="https://discord.com/invite/A59Uxyy2k9" target="_self">Discord community</a> with hundreds of Dataherald AI users for updates, answers to your questions, and other support!</p><p class="slate-p" placeholder="Type / for widgets"></p>

You can interact with the Dataherald API through HTTP requests from any language or through our official Python SDK package.

To install the official Python SDK package, run the following command:

 with hundreds of Dataherald AI users for updates, answers to your questions, and other support!

<p class="slate-p" placeholder="Type / for widgets">The Dataherald API uses API keys for authentication. These API keys are at an organization level. You can visit the API Keys page in the Admin Console and retrieve a key that you can use to make API calls.</p><p class="slate-p" placeholder="Type / for widgets">You should store you API keys securely and do not expose it in client side code. If you feel a key has been exposed please revoke it immediately from the Admin Console and report it to us.</p><p class="slate-p" placeholder="Type / for widgets">All API requests should include your API key in an <code class="slate-code">X-API-Key</code> HTTP header as follows</p><pre class="slate-CodeBlockElement slate-code_block"><div contenteditable="false"><div ><span aria-live="polite" aria-atomic="false" aria-relevant="additions text" ></span><div ><div ><div >Select...</div><div ><div  style="display:inline-block"><input type="text" autoCapitalize="none" autoComplete="off" autoCorrect="off" id="react-select-145-input" spellcheck="false" tabindex="0" value="" aria-autocomplete="list" style="box-sizing:content-box;width:1px;label:input;background:0;border:0;font-size:inherit;opacity:1;outline:0;padding:0;color:inherit"/><div style="position:absolute;top:0;left:0;visibility:hidden;height:0;overflow:scroll;white-space:pre"></div></div></div></div><div ><span ></span><div  aria-hidden="true"><svg height="20" width="20" viewBox="0 0 20 20" aria-hidden="true" focusable="false" ><path d="M4.516 7.548c0.436-0.446 1.043-0.481 1.576 0l3.908 3.747 3.908-3.747c0.533-0.481 1.141-0.446 1.574 0 0.436 0.445 0.408 1.197 0 1.615-0.406 0.418-4.695 4.502-4.695 4.502-0.217 0.223-0.502 0.335-0.787 0.335s-0.57-0.112-0.789-0.335c0 0-4.287-4.084-4.695-4.502s-0.436-1.17 0-1.615z"></path></svg></div></div></div></div></div><code class="keep-markup drop-tokens"><div class="slate-code_line">&apos;X-API-Key: dh-2d3b40……………………………………………………………………60af&apos;</div></code></pre><p class="slate-p" placeholder="Type / for widgets"> </p><p class="slate-p" placeholder="Type / for widgets">For example to make a call to get a list of database connections for your organization the call will be as follows:</p><p class="slate-p" placeholder="Type / for widgets"></p><pre class="slate-CodeBlockElement slate-code_block"><div contenteditable="false"><div ><span aria-live="polite" aria-atomic="false" aria-relevant="additions text" ></span><div ><div ><div >Select...</div><div ><div  style="display:inline-block"><input type="text" autoCapitalize="none" autoComplete="off" autoCorrect="off" id="react-select-146-input" spellcheck="false" tabindex="0" value="" aria-autocomplete="list" style="box-sizing:content-box;width:1px;label:input;background:0;border:0;font-size:inherit;opacity:1;outline:0;padding:0;color:inherit"/><div style="position:absolute;top:0;left:0;visibility:hidden;height:0;overflow:scroll;white-space:pre"></div></div></div></div><div ><span ></span><div  aria-hidden="true"><svg height="20" width="20" viewBox="0 0 20 20" aria-hidden="true" focusable="false" ><path d="M4.516 7.548c0.436-0.446 1.043-0.481 1.576 0l3.908 3.747 3.908-3.747c0.533-0.481 1.141-0.446 1.574 0 0.436 0.445 0.408 1.197 0 1.615-0.406 0.418-4.695 4.502-4.695 4.502-0.217 0.223-0.502 0.335-0.787 0.335s-0.57-0.112-0.789-0.335c0 0-4.287-4.084-4.695-4.502s-0.436-1.17 0-1.615z"></path></svg></div></div></div></div></div><code class="keep-markup drop-tokens"><div class="slate-code_line">curl -X &apos;GET&apos; \</div><div class="slate-code_line">  &apos;https://stage.api.dataherald.ai/api/database-connections&apos; \</div><div class="slate-code_line">  -H &apos;accept: application/json&apos; \</div><div class="slate-code_line">  -H &apos;X-API-Key: dh-2d3b40……………………………………………………………………60af&apos;</div></code></pre><p class="slate-p" placeholder="Type / for widgets">If you are using the Python SDK, you can set your key as an environment variable </p><pre class="slate-CodeBlockElement slate-code_block"><div contenteditable="false"><div ><span aria-live="polite" aria-atomic="false" aria-relevant="additions text" ></span><div ><div ><div >Select...</div><div ><div  style="display:inline-block"><input type="text" autoCapitalize="none" autoComplete="off" autoCorrect="off" id="react-select-147-input" spellcheck="false" tabindex="0" value="" aria-autocomplete="list" style="box-sizing:content-box;width:1px;label:input;background:0;border:0;font-size:inherit;opacity:1;outline:0;padding:0;color:inherit"/><div style="position:absolute;top:0;left:0;visibility:hidden;height:0;overflow:scroll;white-space:pre"></div></div></div></div><div ><span ></span><div  aria-hidden="true"><svg height="20" width="20" viewBox="0 0 20 20" aria-hidden="true" focusable="false" ><path d="M4.516 7.548c0.436-0.446 1.043-0.481 1.576 0l3.908 3.747 3.908-3.747c0.533-0.481 1.141-0.446 1.574 0 0.436 0.445 0.408 1.197 0 1.615-0.406 0.418-4.695 4.502-4.695 4.502-0.217 0.223-0.502 0.335-0.787 0.335s-0.57-0.112-0.789-0.335c0 0-4.287-4.084-4.695-4.502s-0.436-1.17 0-1.615z"></path></svg></div></div></div></div></div><code class="keep-markup drop-tokens"><div class="slate-code_line">export DATAHERALD_API_KEY= dh-2d3b40……………………………………………………………………60af</div></code></pre><p class="slate-p" placeholder="Type / for widgets">While you can pass your API key to the Dataherald Client object as a keyword argument, this is not recommended since your API key will be stored with your source control</p><pre class="slate-CodeBlockElement slate-code_block"><div contenteditable="false"><div ><span aria-live="polite" aria-atomic="false" aria-relevant="additions text" ></span><div ><div ><div >Select...</div><div ><div  style="display:inline-block"><input type="text" autoCapitalize="none" autoComplete="off" autoCorrect="off" id="react-select-148-input" spellcheck="false" tabindex="0" value="" aria-autocomplete="list" style="box-sizing:content-box;width:1px;label:input;background:0;border:0;font-size:inherit;opacity:1;outline:0;padding:0;color:inherit"/><div style="position:absolute;top:0;left:0;visibility:hidden;height:0;overflow:scroll;white-space:pre"></div></div></div></div><div ><span ></span><div  aria-hidden="true"><svg height="20" width="20" viewBox="0 0 20 20" aria-hidden="true" focusable="false" ><path d="M4.516 7.548c0.436-0.446 1.043-0.481 1.576 0l3.908 3.747 3.908-3.747c0.533-0.481 1.141-0.446 1.574 0 0.436 0.445 0.408 1.197 0 1.615-0.406 0.418-4.695 4.502-4.695 4.502-0.217 0.223-0.502 0.335-0.787 0.335s-0.57-0.112-0.789-0.335c0 0-4.287-4.084-4.695-4.502s-0.436-1.17 0-1.615z"></path></svg></div></div></div></div></div><code class="language-python keep-markup drop-tokens"><div class="slate-code_line">dh_client = Dataherald(base_url = &apos;https://api.dataherald.ai&apos;,</div><div class="slate-code_line">                       api_key=&apos;dh-2d3b40……………………………………………………………………60af&apos; ,</div><div class="slate-code_line">                       timeout=360)</div></code></pre><p class="slate-p" placeholder="Type / for widgets"></p>

The Dataherald API uses API keys for authentication. These API keys are at an organization level. You can visit the API Keys page in the Admin Console and retrieve a key that you can use to make API calls.

You should store you API keys securely and do not expose it in client side code. If you feel a key has been exposed please revoke it immediately from the Admin Console and report it to us.

All API requests should include your API key in an 

For example to make a call to get a list of database connections for your organization the call will be as follows:

If you are using the Python SDK, you can set your key as an environment variable 

While you can pass your API key to the Dataherald Client object as a keyword argument, this is not recommended since your API key will be stored with your source control

<p class="slate-p" placeholder="Type / for widgets">Dataherald is a hosted NL-to-SQL API that enables any developer to embed natural language querying to their relational database. You can access Dataherald directly through the <a href="https://docs.dataherald.com/introduction-1" target="_blank">API</a> or through the <a href="https://pypi.org/project/dataherald/" target="_blank">Python SDK</a>.</p><p class="slate-p" placeholder="Type / for widgets">In this guide, we’ll walk through the steps for getting started with the Dataherald Python SDK. You can also follow this <a href="https://www.loom.com/share/29f5944048554cf3969cd00a992f7551?sid=3a85f379-ca22-4962-8dfd-14b19458befc" target="_self">quick 2-minute video guide</a> to get set up.</p><p class="slate-p" placeholder="Type / for widgets">We highly recommend joining our <a href="https://discord.gg/A59Uxyy2k9" target="_self">Discord community</a> for support.</p><div class="slate-hr"><hr contenteditable="false" class="slate-HrElement-hr"/></div><p class="slate-p" placeholder="Type / for widgets"></p><h1 class="slate-h1">Install and Instantiate Dataherald</h1><p class="slate-p" placeholder="Type / for widgets">First, you have to <a href="https://www.dataherald.com/news/introducing-dhai?utm_medium=website&amp;utm_source=docs" target="_self">sign up for an account</a>. Once that is done, you can log into the <a href="https://console.dataherald.ai/" target="_blank">Dataherald admin console</a> and generate an API key. Save this key, as it won’t be recorded anywhere unencrypted.</p><p class="slate-p" placeholder="Type / for widgets">Then you can start building the app. First, install the Dataherald SDK.</p><pre class="slate-CodeBlockElement slate-code_block"><div contenteditable="false"><div ><span aria-live="polite" aria-atomic="false" aria-relevant="additions text" ></span><div ><div ><div >Select...</div><div ><div  style="display:inline-block"><input type="text" autoCapitalize="none" autoComplete="off" autoCorrect="off" id="react-select-246-input" spellcheck="false" tabindex="0" value="" aria-autocomplete="list" style="box-sizing:content-box;width:1px;label:input;background:0;border:0;font-size:inherit;opacity:1;outline:0;padding:0;color:inherit"/><div style="position:absolute;top:0;left:0;visibility:hidden;height:0;overflow:scroll;white-space:pre"></div></div></div></div><div ><span ></span><div  aria-hidden="true"><svg height="20" width="20" viewBox="0 0 20 20" aria-hidden="true" focusable="false" ><path d="M4.516 7.548c0.436-0.446 1.043-0.481 1.576 0l3.908 3.747 3.908-3.747c0.533-0.481 1.141-0.446 1.574 0 0.436 0.445 0.408 1.197 0 1.615-0.406 0.418-4.695 4.502-4.695 4.502-0.217 0.223-0.502 0.335-0.787 0.335s-0.57-0.112-0.789-0.335c0 0-4.287-4.084-4.695-4.502s-0.436-1.17 0-1.615z"></path></svg></div></div></div></div></div><code class="language-python keep-markup drop-tokens"><div class="slate-code_line">!pip install dataherald</div></code></pre><p class="slate-p" placeholder="Type / for widgets">You can then instantiate the Dataherald client. Note that you should <em class="slate-italic">never</em> save your API key with your code. The code below assumes the API key is saved as an environment variable.</p><pre class="slate-CodeBlockElement slate-code_block"><div contenteditable="false"><div ><span aria-live="polite" aria-atomic="false" aria-relevant="additions text" ></span><div ><div ><div >Select...</div><div ><div  style="display:inline-block"><input type="text" autoCapitalize="none" autoComplete="off" autoCorrect="off" id="react-select-247-input" spellcheck="false" tabindex="0" value="" aria-autocomplete="list" style="box-sizing:content-box;width:1px;label:input;background:0;border:0;font-size:inherit;opacity:1;outline:0;padding:0;color:inherit"/><div style="position:absolute;top:0;left:0;visibility:hidden;height:0;overflow:scroll;white-space:pre"></div></div></div></div><div ><span ></span><div  aria-hidden="true"><svg height="20" width="20" viewBox="0 0 20 20" aria-hidden="true" focusable="false" ><path d="M4.516 7.548c0.436-0.446 1.043-0.481 1.576 0l3.908 3.747 3.908-3.747c0.533-0.481 1.141-0.446 1.574 0 0.436 0.445 0.408 1.197 0 1.615-0.406 0.418-4.695 4.502-4.695 4.502-0.217 0.223-0.502 0.335-0.787 0.335s-0.57-0.112-0.789-0.335c0 0-4.287-4.084-4.695-4.502s-0.436-1.17 0-1.615z"></path></svg></div></div></div></div></div><code class="language-python keep-markup drop-tokens"><div class="slate-code_line">import os</div><div class="slate-code_line">from dataherald import Dataherald</div><div class="slate-code_line"></div><div class="slate-code_line">API_KEY = os.environ[&apos;API_KEY&apos;]</div><div class="slate-code_line">dh_client = Dataherald(api_key=API_KEY)</div></code></pre><h1 class="slate-h1">Create a Database Connection</h1><p class="slate-p" placeholder="Type / for widgets">The next step is to give the tool access to your relational data warehouse. Dataherald uses a standard ODBC protocol to connect to the relational data warehouse of your choice. It currently supports:</p><ul class="slate-ul"><li class="slate-li"><div style="position:relative">PostGres</div></li><li class="slate-li"><div style="position:relative">Databricks</div></li><li class="slate-li"><div style="position:relative">Snowflake</div></li><li class="slate-li"><div style="position:relative">BigQuery</div></li><li class="slate-li"><div style="position:relative">AWS Athena</div></li><li class="slate-li"><div style="position:relative">MotherDuck</div></li></ul><p class="slate-p" placeholder="Type / for widgets">In order to create a database connection, all you need is a connection URI for your database. Once you have that, you can create the database connection from the <a href="https://console.dataherald.ai/databases" target="_blank">admin console’s Databases tab</a> or through the SDK.</p><p class="slate-p" placeholder="Type / for widgets">For additional instructions on generating the connection URI (such as using an SSH connection or if your warehouse requires additional credentials like BigQuery) please refer to this <a href="https://docs.dataherald.com/database-connection/add-database-connection" target="_blank">page</a>.</p><div style="text-align:center"><img src="https://miro.medium.com/v2/resize:fit:1400/1*1V4P057uaT4iPBiIJxKXqQ.png" alt="" style="width:60%"/></div><p class="slate-p slate-indent-0" style="margin-left:0px" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets">To create a database connection using the SDK use the following command:</p><pre class="slate-CodeBlockElement slate-code_block"><div contenteditable="false"><div ><span aria-live="polite" aria-atomic="false" aria-relevant="additions text" ></span><div ><div ><div >Select...</div><div ><div  style="display:inline-block"><input type="text" autoCapitalize="none" autoComplete="off" autoCorrect="off" id="react-select-248-input" spellcheck="false" tabindex="0" value="" aria-autocomplete="list" style="box-sizing:content-box;width:1px;label:input;background:0;border:0;font-size:inherit;opacity:1;outline:0;padding:0;color:inherit"/><div style="position:absolute;top:0;left:0;visibility:hidden;height:0;overflow:scroll;white-space:pre"></div></div></div></div><div ><span ></span><div  aria-hidden="true"><svg height="20" width="20" viewBox="0 0 20 20" aria-hidden="true" focusable="false" ><path d="M4.516 7.548c0.436-0.446 1.043-0.481 1.576 0l3.908 3.747 3.908-3.747c0.533-0.481 1.141-0.446 1.574 0 0.436 0.445 0.408 1.197 0 1.615-0.406 0.418-4.695 4.502-4.695 4.502-0.217 0.223-0.502 0.335-0.787 0.335s-0.57-0.112-0.789-0.335c0 0-4.287-4.084-4.695-4.502s-0.436-1.17 0-1.615z"></path></svg></div></div></div></div></div><code class="language-python keep-markup drop-tokens"><div class="slate-code_line">alias = &quot;&lt;Insert a human-readable name for the connection&gt;&quot;</div><div class="slate-code_line">use_ssh = False # Dataherald provides support for SSH connections</div><div class="slate-code_line">connection_uri = &quot;&lt;Insert connection URI here for your data warehouse&gt;&quot;</div><div class="slate-code_line"></div><div class="slate-code_line">db_connection = dh_client.database_connections.create(                  </div><div class="slate-code_line">  alias=alias,                  </div><div class="slate-code_line">  use_ssh=use_ssh,                  </div><div class="slate-code_line">  connection_uri=connection_uri</div><div class="slate-code_line">)</div></code></pre><h1 class="slate-h1">Scan Tables</h1><p class="slate-p" placeholder="Type / for widgets">The next step is to scan the tables in the database which allows Dataherald to build context about the schema and dataset. During the scan Dataherald will:</p><ul class="slate-ul"><li class="slate-li"><div style="position:relative">Read the schema to collect the table and column schema including data types</div></li><li class="slate-li"><div style="position:relative">Identify categorical columns and their values</div></li><li class="slate-li"><div style="position:relative">Determine relationships between tables</div></li><li class="slate-li"><div style="position:relative">and more</div></li></ul><p class="slate-p" placeholder="Type / for widgets">You can specify tables to be scanned or just have the tool scan all available tables. Please note this is an asynchronous process and can take some time to complete.</p><pre class="slate-CodeBlockElement slate-code_block"><div contenteditable="false"><div ><span aria-live="polite" aria-atomic="false" aria-relevant="additions text" ></span><div ><div ><div >Select...</div><div ><div  style="display:inline-block"><input type="text" autoCapitalize="none" autoComplete="off" autoCorrect="off" id="react-select-249-input" spellcheck="false" tabindex="0" value="" aria-autocomplete="list" style="box-sizing:content-box;width:1px;label:input;background:0;border:0;font-size:inherit;opacity:1;outline:0;padding:0;color:inherit"/><div style="position:absolute;top:0;left:0;visibility:hidden;height:0;overflow:scroll;white-space:pre"></div></div></div></div><div ><span ></span><div  aria-hidden="true"><svg height="20" width="20" viewBox="0 0 20 20" aria-hidden="true" focusable="false" ><path d="M4.516 7.548c0.436-0.446 1.043-0.481 1.576 0l3.908 3.747 3.908-3.747c0.533-0.481 1.141-0.446 1.574 0 0.436 0.445 0.408 1.197 0 1.615-0.406 0.418-4.695 4.502-4.695 4.502-0.217 0.223-0.502 0.335-0.787 0.335s-0.57-0.112-0.789-0.335c0 0-4.287-4.084-4.695-4.502s-0.436-1.17 0-1.615z"></path></svg></div></div></div></div></div><code class="language-python keep-markup drop-tokens"><div class="slate-code_line">table_names = [ &lt;List of tables to scan&gt; ]</div><div class="slate-code_line"></div><div class="slate-code_line">dh_client.table_descriptions.sync_schemas(  </div><div class="slate-code_line">  db_connection_id=db_connection.id,  </div><div class="slate-code_line">  table_names=table_names</div><div class="slate-code_line">)</div></code></pre><h1 class="slate-h1">Ask a Question</h1><p class="slate-p" placeholder="Type / for widgets">The engine is finally set up to query the database in natural language. There are three main resources involved in querying the database in natural language:</p><ul class="slate-ul"><li class="slate-li"><div style="position:relative"><strong class="slate-bold">Prompts: </strong>the natural language text or question</div></li><li class="slate-li"><div style="position:relative"><strong class="slate-bold">SQL-Generations:</strong> the AI generated SQL query to answer the natural language prompt</div></li><li class="slate-li"><div style="position:relative"><strong class="slate-bold">NL-Generations:</strong> a natural language answer to the prompt</div></li></ul><p class="slate-p" placeholder="Type / for widgets">While these resources are linked and build on each other (ie you need to create a Prompt before a SQL-Generation before an NL-Generation), the API and SDK provide methods where you can create these in a single call.</p><p class="slate-p" placeholder="Type / for widgets">The following example creates a SQL query for a given question.</p><pre class="slate-CodeBlockElement slate-code_block"><div contenteditable="false"><div ><span aria-live="polite" aria-atomic="false" aria-relevant="additions text" ></span><div ><div ><div >Select...</div><div ><div  style="display:inline-block"><input type="text" autoCapitalize="none" autoComplete="off" autoCorrect="off" id="react-select-250-input" spellcheck="false" tabindex="0" value="" aria-autocomplete="list" style="box-sizing:content-box;width:1px;label:input;background:0;border:0;font-size:inherit;opacity:1;outline:0;padding:0;color:inherit"/><div style="position:absolute;top:0;left:0;visibility:hidden;height:0;overflow:scroll;white-space:pre"></div></div></div></div><div ><span ></span><div  aria-hidden="true"><svg height="20" width="20" viewBox="0 0 20 20" aria-hidden="true" focusable="false" ><path d="M4.516 7.548c0.436-0.446 1.043-0.481 1.576 0l3.908 3.747 3.908-3.747c0.533-0.481 1.141-0.446 1.574 0 0.436 0.445 0.408 1.197 0 1.615-0.406 0.418-4.695 4.502-4.695 4.502-0.217 0.223-0.502 0.335-0.787 0.335s-0.57-0.112-0.789-0.335c0 0-4.287-4.084-4.695-4.502s-0.436-1.17 0-1.615z"></path></svg></div></div></div></div></div><code class="language-python keep-markup drop-tokens"><div class="slate-code_line">from dataherald.types.sql_generation_create_params import Prompt</div><div class="slate-code_line"></div><div class="slate-code_line">prompt = Prompt(text=&quot;&lt;Insert Question here&gt;&quot;,                 </div><div class="slate-code_line">db_connection_id=&quot;&lt;Insert Database Connection ID here&gt;&quot;)</div><div class="slate-code_line"></div><div class="slate-code_line">response = dh_client.sql_generations.create(prompt=prompt)</div></code></pre><p class="slate-p" placeholder="Type / for widgets">To access the generated SQL:</p><pre class="slate-CodeBlockElement slate-code_block"><div contenteditable="false"><div ><span aria-live="polite" aria-atomic="false" aria-relevant="additions text" ></span><div ><div ><div >Select...</div><div ><div  style="display:inline-block"><input type="text" autoCapitalize="none" autoComplete="off" autoCorrect="off" id="react-select-251-input" spellcheck="false" tabindex="0" value="" aria-autocomplete="list" style="box-sizing:content-box;width:1px;label:input;background:0;border:0;font-size:inherit;opacity:1;outline:0;padding:0;color:inherit"/><div style="position:absolute;top:0;left:0;visibility:hidden;height:0;overflow:scroll;white-space:pre"></div></div></div></div><div ><span ></span><div  aria-hidden="true"><svg height="20" width="20" viewBox="0 0 20 20" aria-hidden="true" focusable="false" ><path d="M4.516 7.548c0.436-0.446 1.043-0.481 1.576 0l3.908 3.747 3.908-3.747c0.533-0.481 1.141-0.446 1.574 0 0.436 0.445 0.408 1.197 0 1.615-0.406 0.418-4.695 4.502-4.695 4.502-0.217 0.223-0.502 0.335-0.787 0.335s-0.57-0.112-0.789-0.335c0 0-4.287-4.084-4.695-4.502s-0.436-1.17 0-1.615z"></path></svg></div></div></div></div></div><code class="language-python keep-markup drop-tokens"><div class="slate-code_line">print(response.sql)</div></code></pre><h1 class="slate-h1"></h1><p class="slate-p" placeholder="Type / for widgets"></p>

Dataherald is a hosted NL-to-SQL API that enables any developer to embed natural language querying to their relational database. You can access Dataherald directly through the 

In this guide, we’ll walk through the steps for getting started with the Dataherald Python SDK. You can also follow this 

. Once that is done, you can log into the 

 and generate an API key. Save this key, as it won’t be recorded anywhere unencrypted.

Then you can start building the app. First, install the Dataherald SDK.

You can then instantiate the Dataherald client. Note that you should 

 save your API key with your code. The code below assumes the API key is saved as an environment variable.

The next step is to give the tool access to your relational data warehouse. Dataherald uses a standard ODBC protocol to connect to the relational data warehouse of your choice. It currently supports:

In order to create a database connection, all you need is a connection URI for your database. Once you have that, you can create the database connection from the 

For additional instructions on generating the connection URI (such as using an SSH connection or if your warehouse requires additional credentials like BigQuery) please refer to this 

To create a database connection using the SDK use the following command:

The next step is to scan the tables in the database which allows Dataherald to build context about the schema and dataset. During the scan Dataherald will:

You can specify tables to be scanned or just have the tool scan all available tables. Please note this is an asynchronous process and can take some time to complete.

The engine is finally set up to query the database in natural language. There are three main resources involved in querying the database in natural language:

While these resources are linked and build on each other (ie you need to create a Prompt before a SQL-Generation before an NL-Generation), the API and SDK provide methods where you can create these in a single call.

The following example creates a SQL query for a given question.

<p class="slate-p" placeholder="Type / for widgets">For technical support, join our <a href="https://discord.gg/A59Uxyy2k9" target="_self">Discord community</a>! With hundreds of active users and Dataherald support active, the Discord community is the best place to get quick answers to questions about using DHAI.</p><p class="slate-p" placeholder="Type / for widgets">For billing support, contact <a href="mailto:support@dataherald.com" target="_blank">support@dataherald.com</a>.</p><p class="slate-p" placeholder="Type / for widgets"></p>

! With hundreds of active users and Dataherald support active, the Discord community is the best place to get quick answers to questions about using DHAI.

<p class="slate-p" placeholder="Type / for widgets">All Dataherald resources, including database-connections, sql-generation and nl-generation have a <code class="slate-code">metadata</code> parameter. You can use this parameter to attach key-value pairs to Dataherald objects.</p><p class="slate-p" placeholder="Type / for widgets">You can use the metada to store additional application specific information on an object. For example, you could store user&apos;s name that initiated the query or store the corresponding unique identifier from your system for the Dataherald object. The Dataherald backend does not do anything with the metadata other than storing and returning it.</p><p class="slate-p" placeholder="Type / for widgets">The following API call stores the key value pair along with sql-generation object</p><pre class="slate-CodeBlockElement slate-code_block"><div contenteditable="false"><div ><span aria-live="polite" aria-atomic="false" aria-relevant="additions text" ></span><div ><div ><div >Select...</div><div ><div  style="display:inline-block"><input type="text" autoCapitalize="none" autoComplete="off" autoCorrect="off" id="react-select-21-input" spellcheck="false" tabindex="0" value="" aria-autocomplete="list" style="box-sizing:content-box;width:1px;label:input;background:0;border:0;font-size:inherit;opacity:1;outline:0;padding:0;color:inherit"/><div style="position:absolute;top:0;left:0;visibility:hidden;height:0;overflow:scroll;white-space:pre"></div></div></div></div><div ><span ></span><div  aria-hidden="true"><svg height="20" width="20" viewBox="0 0 20 20" aria-hidden="true" focusable="false" ><path d="M4.516 7.548c0.436-0.446 1.043-0.481 1.576 0l3.908 3.747 3.908-3.747c0.533-0.481 1.141-0.446 1.574 0 0.436 0.445 0.408 1.197 0 1.615-0.406 0.418-4.695 4.502-4.695 4.502-0.217 0.223-0.502 0.335-0.787 0.335s-0.57-0.112-0.789-0.335c0 0-4.287-4.084-4.695-4.502s-0.436-1.17 0-1.615z"></path></svg></div></div></div></div></div><code class="keep-markup drop-tokens"><div class="slate-code_line">curl -X &apos;POST&apos; \</div><div class="slate-code_line">  &apos;https://api.dataherald.ai/api/prompts/sql-generations&apos; \</div><div class="slate-code_line">  -H &apos;accept: application/json&apos; \</div><div class="slate-code_line">  -H &apos;X-API-Key: dh-2d3…………………………………………………………………………………………………………..af&apos; \</div><div class="slate-code_line">  -H &apos;Content-Type: application/json&apos; \</div><div class="slate-code_line">  -d &apos;{</div><div class="slate-code_line">  &quot;finetuning_id&quot;: &quot;6597665e4641238c6e9cb&quot;,</div><div class="slate-code_line">  &quot;metadata&quot;: {&quot;user&quot;: &quot;Jane Doe&quot;},</div><div class="slate-code_line">  &quot;prompt&quot;: {</div><div class="slate-code_line">    &quot;text&quot;: &quot;Los Angeles Home price trends&quot;,</div><div class="slate-code_line">    &quot;db_connection_id&quot;: &quot;6596fd97c97ab8dasdfrs5&quot;</div><div class="slate-code_line">   </div><div class="slate-code_line">  }</div><div class="slate-code_line">}&apos;</div></code></pre><p class="slate-p" placeholder="Type / for widgets">To do the same through the Python SDK</p><pre class="slate-CodeBlockElement slate-code_block"><div contenteditable="false"><div ><span aria-live="polite" aria-atomic="false" aria-relevant="additions text" ></span><div ><div ><div >Select...</div><div ><div  style="display:inline-block"><input type="text" autoCapitalize="none" autoComplete="off" autoCorrect="off" id="react-select-22-input" spellcheck="false" tabindex="0" value="" aria-autocomplete="list" style="box-sizing:content-box;width:1px;label:input;background:0;border:0;font-size:inherit;opacity:1;outline:0;padding:0;color:inherit"/><div style="position:absolute;top:0;left:0;visibility:hidden;height:0;overflow:scroll;white-space:pre"></div></div></div></div><div ><span ></span><div  aria-hidden="true"><svg height="20" width="20" viewBox="0 0 20 20" aria-hidden="true" focusable="false" ><path d="M4.516 7.548c0.436-0.446 1.043-0.481 1.576 0l3.908 3.747 3.908-3.747c0.533-0.481 1.141-0.446 1.574 0 0.436 0.445 0.408 1.197 0 1.615-0.406 0.418-4.695 4.502-4.695 4.502-0.217 0.223-0.502 0.335-0.787 0.335s-0.57-0.112-0.789-0.335c0 0-4.287-4.084-4.695-4.502s-0.436-1.17 0-1.615z"></path></svg></div></div></div></div></div><code class="language-python keep-markup drop-tokens"><div class="slate-code_line">import dataherald</div><div class="slate-code_line"></div><div class="slate-code_line">dh_client = Dataherald(timeout=360)</div><div class="slate-code_line">prompt = Prompt(text=&apos;Los Angeles Home price trends&apos;, db_connection_id=&apos;6596fd97c97ab8dasdfrs5&apos;)</div><div class="slate-code_line">dh_client.sql_generations.create(prompt=prompt, metadata={&apos;user&apos;:&apos;Jane Doe&apos;})</div></code></pre><p class="slate-p" placeholder="Type / for widgets"></p>

All Dataherald resources, including database-connections, sql-generation and nl-generation have a 

 parameter. You can use this parameter to attach key-value pairs to Dataherald objects.

You can use the metada to store additional application specific information on an object. For example, you could store user's name that initiated the query or store the corresponding unique identifier from your system for the Dataherald object. The Dataherald backend does not do anything with the metadata other than storing and returning it.

The following API call stores the key value pair along with sql-generation object

<p class="slate-p" placeholder="Type / for widgets"><p class="slate-p" placeholder="Type / for widgets">The Database Connection object defines connections between Dataherald and your relational database. Through the APIs and SDK methods that interact with this resource you can create connections to new databases providing necessary authentication information, including SSH information if a tunnel needs to set up to access the database. </p><p class="slate-p" placeholder="Type / for widgets">We currently provide support for connections to <code class="slate-code">Postegres</code>, <code class="slate-code">Databricks</code>, <code class="slate-code">BigQuery</code>,<code class="slate-code"> Snowflake</code> and <code class="slate-code">MotherDuck</code>. Since the Dataherald engine uses SQLAlchemy and an ODBC driver to connect to your database, other databases might work as well, though they are not officially supported.</p><p class="slate-p" placeholder="Type / for widgets"></p></p>

<p class="slate-p" placeholder="Type / for widgets">Golden SQL are sample natural language to SQL pairs for a given database. These can be used as training data to finetune  a model, or as samples to be passed to the LLM in few-shot prompting.</p><p class="slate-p" placeholder="Type / for widgets"></p>

Golden SQL are sample natural language to SQL pairs for a given database. These can be used as training data to finetune  a model, or as samples to be passed to the LLM in few-shot prompting.

<p class="slate-p" placeholder="Type / for widgets">SQL Generation is the main object that represents a translation of a natural language question to SQL for a given relational database given the context that was added to the Dataherald engine. Other than metadata, these objects are immutable and cannot be modified. SQL Generation objects are created from a Prompt object and there can be multiple SQL Generation objects for a single Prompt object.</p><p class="slate-p" placeholder="Type / for widgets">While SQL Generations can only be created for a Prompt object, you can create the SQL Generation and Prompt in a single call to the Dataherald Engine.</p><p class="slate-p" placeholder="Type / for widgets">For best results a SQL Generation should be created with a fine-tuned model. However, you can still create a SQL Generation without a fine-tuned model, in which case the agent will use relevant Golden SQL for few shot prompting. </p><p class="slate-p" placeholder="Type / for widgets"></p>

SQL Generation is the main object that represents a translation of a natural language question to SQL for a given relational database given the context that was added to the Dataherald engine. Other than metadata, these objects are immutable and cannot be modified. SQL Generation objects are created from a Prompt object and there can be multiple SQL Generation objects for a single Prompt object.

While SQL Generations can only be created for a Prompt object, you can create the SQL Generation and Prompt in a single call to the Dataherald Engine.

For best results a SQL Generation should be created with a fine-tuned model. However, you can still create a SQL Generation without a fine-tuned model, in which case the agent will use relevant Golden SQL for few shot prompting. 

<p class="slate-p" placeholder="Type / for widgets"><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets">After successfully creating a Database Connection, you need to upload the schema into Dataherald you can start generating SQL from natural language. The Dataherald engine provides a scanner which can automatically scan the database and retrieve the following:</p><ul class="slate-ul"><li class="slate-li">Table and column schema</li><li class="slate-li">Identify low cardinality columns and their values</li><li class="slate-li">Mine SQL query history for synthetic Golden SQL generation</li></ul><p class="slate-p" placeholder="Type / for widgets">The information in the table descriptions are used in during fine-tuning and SQL generation Schema Linking step (where the agent identifies the relevant tables and columns to use in the query) and in the SQL generation pipeline. In addition to information collected automatically in the scan, you can manually add descriptions to tables and columns. </p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p></p>

<p class="slate-p" placeholder="Type / for widgets">In order to achieve best NL-to-SQL performance, you need to fine-tune a model specific to your own relational data. The Finetuning object represents a model that has been fine-tuned from a set of Golden SQL objects. Once a Finetuning object is succesfully created, you can use it in creating SQL Generation and NL Generation objects.</p><p class="slate-p" placeholder="Type / for widgets">While you can create a Finetuning object from as little as 10 Golden SQL, for the finetuned model to yield accurate NL-to-SQL the training set will need at least hundreds of samples with covering various tables in the relational database.</p><p class="slate-p" placeholder="Type / for widgets"></p>

In order to achieve best NL-to-SQL performance, you need to fine-tune a model specific to your own relational data. The Finetuning object represents a model that has been fine-tuned from a set of Golden SQL objects. Once a Finetuning object is succesfully created, you can use it in creating SQL Generation and NL Generation objects.

While you can create a Finetuning object from as little as 10 Golden SQL, for the finetuned model to yield accurate NL-to-SQL the training set will need at least hundreds of samples with covering various tables in the relational database.

<p class="slate-p" placeholder="Type / for widgets">A Prompt object represents a natural language question from a given relational database. Prompt objects are immutable and cannot be modified once created. SQL Generations and NL Generations are created from Prompt objects. </p>

A Prompt object represents a natural language question from a given relational database. Prompt objects are immutable and cannot be modified once created. SQL Generations and NL Generations are created from Prompt objects. 

<p class="slate-p" placeholder="Type / for widgets">Use the DHAI agent to generate a SQL query to answer the question posed in the Prompt. This endpoint will also create a Prompt object.</p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p>

Use the DHAI agent to generate a SQL query to answer the question posed in the Prompt. This endpoint will also create a Prompt object.

The ID of the fine-tuning process associated with the SQL generation.

finetuning_id

The evaluation data used in the request body.

evaluate

If provided, tool will not generate SQL and will instead validate the provided SQL.

Additional metadata associated with the SQL generation.

metadata

prompt

low_latency_mode

The API key used for authentication in the request header.

X-API-Key

The unique identifier of the prompt for which the SQL generation is being created.

The ID of the prompt associated with the SQL generation.

prompt_id

The SQL data used for SQL generation in the request body.

status

The confidence score of the SQL generation result.

confidence_score

The error message returned in the response, if any.

error

The timestamp indicating when the SQL generation was completed.

completed_at

The timestamp indicating when the SQL generation was created.

created_at

The detailed information returned in the response.

detail

<p class="slate-p" placeholder="Type / for widgets">Use the DHAI agent to generate a SQL query to answer the question posed in the Prompt.</p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p>

Use the DHAI agent to generate a SQL query to answer the question posed in the Prompt.

The finetuning_id parameter represents the ID of the fine-tuning configuration to be used for the prompt SQL generation. It should be provided as a string in the request body.

The evaluate parameter represents whether the prompt SQL generation should be evaluated or not. It should be provided as a boolean value (true or false) in the request body.

The metadata parameter represents additional metadata or contextual information related to the prompt SQL generation. It should be provided as a string or JSON object in the request body.

The prompt parameter represents the prompt text used for the SQL generation. It should be provided as a string in the request body.

The X-API-Key parameter represents the API key used for authentication and authorization. It should be provided as a header parameter in the request.

The id parameter is returned in the response and represents the unique identifier of the prompt SQL generation. It can be a string.

The prompt_id parameter is returned in the response and represents the unique identifier of the prompt SQL generation. It can be a string.

The sql parameter represents the generated SQL code for the prompt. It should be provided as a string in the request body.

The confidence_score parameter is returned in the response and represents the confidence score or reliability of the prompt SQL generation. It can be a number.

The error parameter is returned in the response and indicates if there was an error during the prompt SQL generation. It can be a string or null.

The completed_at parameter is returned in the response and represents the timestamp when the prompt SQL generation was completed. It can be a string or null.

The created_at parameter is returned in the response and represents the timestamp when the prompt SQL generation was created. It can be a string.

The detail parameter is returned in the response and contains detailed information about the prompt SQL generation. It can be a string or null.

<p class="slate-p" placeholder="Type / for widgets">Retrieve a specific SQL generation given its <code class="slate-code">id</code>.</p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p>

Retrieve a specific SQL generation given its 

<p class="slate-p" placeholder="Type / for widgets">List all SQL generations in the engine. Includes pagination and ordering arguments.</p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p>

List all SQL generations in the engine. Includes pagination and ordering arguments.

page

page_size

order

ascend

The API key used for authentication and authorization purposes.

The unique identifier for the API response.

The unique identifier for the prompt used in the SQL generation process.

The identifier for the specific finetuning process used in the SQL generation.

The generated SQL code based on the provided parameters and query.

The confidence score indicating the level of confidence in the generated SQL code.

An error message indicating any issues encountered while processing the API request.

The timestamp indicating when the SQL generation process was completed.

The timestamp indicating when the API request was created.

Additional metadata associated with the API response.

Additional details or information related to the API response.

<p class="slate-p" placeholder="Type / for widgets">List all SQL generations in the engine for a given Prompt ID.</p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p>

List all SQL generations in the engine for a given Prompt ID.

This header parameter is used to authenticate the API request. It should contain a valid API key provided by Dataherald AI.

This path parameter represents the unique identifier of the prompt for which SQL generations are requested.

This parameter specifies the level of detail in the response. Possible values include "basic" and "full".

<p class="slate-p" placeholder="Type / for widgets">Execute a SQL query from a given SQL generation ID against the database connection. Includes a <code class="slate-code">max_rows</code> argument to limit the size of the returned data.</p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p>

Execute a SQL query from a given SQL generation ID against the database connection. Includes a 

 argument to limit the size of the returned data.

max_rows

This header parameter is used to authenticate the API request. You need to provide your API key in this parameter to authorize access to the requested resource.

This path parameter represents the unique identifier of the SQL generation to be executed. You need to provide the corresponding ID in the API URL to retrieve the specific SQL generation for execution.

This parameter is used to specify the level of detail in the response. It allows you to control the amount of information returned in the API response.

<p class="slate-p" placeholder="Type / for widgets">A NL Generation object is a natural language answer to a given Prompt object from the associated relational database. An NL Generation is created from an SQL Generation object from the query results after executing the query.</p><p class="slate-p" placeholder="Type / for widgets">There can be multiple NL Generations for each SQL Generation (and these values can be different) as the underlying data can change. While a Prompt object and SQL Generation object must exist in order to create a NL Generation object, all three can be created with a single call to the Dataherald Engine. </p><p class="slate-p" placeholder="Type / for widgets"></p>

A NL Generation object is a natural language answer to a given Prompt object from the associated relational database. An NL Generation is created from an SQL Generation object from the query results after executing the query.

There can be multiple NL Generations for each SQL Generation (and these values can be different) as the underlying data can change. While a Prompt object and SQL Generation object must exist in order to create a NL Generation object, all three can be created with a single call to the Dataherald Engine. 

<p class="slate-p" placeholder="Type / for widgets">This section contains guides on how to set up and get the best NL-to-SQL performance for your database.</p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p>

This section contains guides on how to set up and get the best NL-to-SQL performance for your database.

Sections