# DBT Core

OvalEdge uses an **S3 Bucket** or **NFS Path** to connect to the data source, which allows users to crawl manifest files and build lineage.

<br>

<figure><img src="https://content.gitbook.com/content/hTnkoJQml0pok9awFDhx/blobs/Nl9fDz1tSam01eiJuvje/DBT%20core.png" alt=""><figcaption></figcaption></figure>

## **Overview**

### **Connector Details**

| Connector Category                                                      | ETL System         |
| ----------------------------------------------------------------------- | ------------------ |
| OvalEdge Release Current Connector Version                              | 6.3.4              |
| <p>Connectivity</p><p><em>\[How OvalEdge connects to DBT Core]</em></p> | S3 Bucket/NFS      |
| OvalEdge Releases Supported (Available from)                            | Release6.x Onwards |

### **Connector Features**

| Crawling of Metadata Objects          | Supported                                              |
| ------------------------------------- | ------------------------------------------------------ |
| Metadata Preview                      | Supported                                              |
| Lineage                               | Supported                                              |
| Lineage                               | **Lineage Levels Supported**                           |
| Lineage                               | <ul><li>Table Lineage</li><li>Column Lineage</li></ul> |
| Authentication via Credential Manager | Supported                                              |
| Data Quality                          | Not Supported                                          |
| DAM (Data Access Management)          | Not Supported                                          |
| Bridge                                | Supported                                              |

## **Getting Ready to Establish a Connection**

### **Prerequisites**

The following are the prerequisites required for establishing a connection:&#x20;

### **Service Account User Permissions**

{% hint style="success" %}
***Important:** We recommend having a separate service account to establish a connection from OvalEdge to the data source with minimal permissions.*
{% endhint %}

| **Operations**        | **Minimum Permissions** |
| --------------------- | ----------------------- |
| Connection Validation | Read Only Access        |
| Crawl Objects         | Read Only Access        |

### **Setup a Connection**

{% hint style="warning" %}
***Important**: You must have the Connector Creator role to set up a connection in OvalEdge.*
{% endhint %}

1. Log into OvalEdge, go to Administration > Connectors, click **+ (New Connector)**, search for **DBT Core**, and complete the specific parameters.

{% hint style="success" %}
***Note:** Fields marked with an asterisk (**\***) are mandatory for establishing a connection.*
{% endhint %}

<table data-header-hidden><thead><tr><th width="220.25"></th><th></th></tr></thead><tbody><tr><td><strong>Field Name</strong></td><td><strong>Description</strong></td></tr><tr><td>Connector Type</td><td>By default, "DBT Core" is displayed as the selected connector type.</td></tr><tr><td><strong>Connector Settings</strong></td><td></td></tr><tr><td>Credential Manager<strong>*</strong></td><td><p>Select the desired credentials manager from the dropdown list. Relevant parameters will be displayed based on your selection.</p><p>Supported Credential Managers:</p><ul><li>OE Credential Manager</li><li>HashiCorp Vault</li><li>AWS Secrets Manager</li><li>Azure Key Vault</li></ul></td></tr><tr><td>License Add Ons</td><td><p>OvalEdge connectors have a default license add-on for data crawling and profiling.</p><ul><li>Select the checkbox for <strong>Auto Lineage Add-On</strong> to build data lineage automatically.</li></ul></td></tr><tr><td>Connector Environment</td><td>Select the environment (Example: PROD, STG) configured for the connector.</td></tr><tr><td>Connector Name<strong>*</strong></td><td><p>Enter a unique name for the DBT Core connection.             </p><p>(Example: "DBTCore_Prod").</p></td></tr><tr><td>DBT Core Repository type*</td><td><p>Choose the appropriate option from the drop-down list.</p><ul><li>NFS</li><li>S3</li></ul><p>If you select <strong>NFS</strong> from the drop-down list, enter the following:</p><ul><li><strong>Path*</strong>: Enter the path where the DBT Project resides.</li></ul><p>If you select <strong>S3</strong> from the drop-down list, enter the following:</p><ul><li><strong>Authentication*</strong>: </li><li><p> <strong>IAM user authentication</strong> in Amazon S3 securely controls access to AWS services using AWS Identity and Access Management (IAM).</p><ul><li><strong>Role-based authentication (RBAC)</strong> in Amazon S3 grants access to an S3 bucket using an AWS role instead of access/secret keys.</li></ul></li></ul><p><strong>Note:</strong> Role-based Authentication is not supported in DBT Core for S3.</p><ul><li><strong>S3 Bucket name*:</strong> Enter the name of the S3 bucket where DBT resides.</li><li><strong>S3 zip pathname name:</strong> Enter the path name where the zip file was extracted from the manifest file in the project folder.</li><li><strong>Access key*:</strong> Enter a unique identifier that is part of the credential pair, like a username.<br><strong>Note:</strong> This Field is available when the Authentication mechanism is selected as "IAM User Authentication."</li><li><strong>Secret key*</strong>: A secret, like a password, is used to sign requests to AWS.<br><strong>Note:</strong> This Field is available when the Authentication mechanism is selected as "IAM User Authentication."</li><li><strong>Filter by tags:</strong> Specify tags as filters to limit the scope of objects the connector will interact with.<br><strong>Example:</strong> ovaledge=9,region=ohio,region=oregon</li><li><strong>Region:</strong> The Region refers to the specific geographical location where your Amazon S3 bucket resides. <br><strong>Example:</strong> us-east-1</li><li><strong>SSO Connection Id:</strong> The SSO connection ID is used to authenticate and authorize access to Amazon S3.</li><li><strong>SSO Application Id:</strong> This specific application ID within the SSO system connects to Amazon S3.</li><li><strong>SSO Role Prefix:</strong> Specifies the roles in the SSO system that have access to Amazon S3 and manage permissions.</li><li><strong>RDAM Policy Folder Path:</strong> Enter the Bucket/Folder path in the S3 to write the policies.</li></ul></td></tr><tr><td><strong>Default Governance Roles</strong></td><td></td></tr><tr><td>Default Governance Roles<strong>*</strong></td><td>Select the appropriate users or teams for each governance role from the dropdown list. All users and teams configured in OvalEdge Security are displayed for selection.</td></tr><tr><td><strong>Admin Roles</strong></td><td></td></tr><tr><td>Admin Roles<strong>*</strong></td><td>Select one or more users from the dropdown list for Integration Admin and Security and Governance Admin. All users configured in OvalEdge Security are available for selection.</td></tr><tr><td>No Of Archive Objects<strong>*</strong></td><td><p>It indicates the number of recent metadata changes to a dataset at the source. By default, it is off. You can enable it by toggling the <strong>Archive</strong> button and specifying the number of objects to archive.</p><p><strong>Example:</strong> Setting it to 4 retrieves the last four changes, shown in the 'version' column of the 'Metadata Changes' module.</p></td></tr><tr><td><strong>Bridge</strong></td><td></td></tr><tr><td>Select Bridge<strong>*</strong></td><td><p><strong>If applicable,</strong> select the bridge from the drop-down list.</p><p>The drop-down list displays all active bridges configured in OvalEdge. These bridges enable communication between data sources and OvalEdge without altering firewall rules.</p></td></tr></tbody></table>

2. After entering all connection details, you can perform the following actions:
   1. Click **Validate** to verify the connection.
   2. Click **Save** to store the connection for future use.
   3. Click **Save & Configure** to apply [additional settings](https://docs.google.com/document/d/1ce8buHSDcbdcmlVkOgClbJPzieiHdBvYHhFFIURGQZA/edit#heading=h.f0c3vossfj57) before saving.
3. The saved connection will appear on the Connectors home page.

## **Connectivity Troubleshooting**

If incorrect parameters are provided, you may encounter error messages. To resolve these issues, ensure all input is correct. If problems persist, contact your assigned **OvalEdge** support team.

<table data-header-hidden><thead><tr><th width="94"></th><th></th><th></th></tr></thead><tbody><tr><td><strong>S. No.</strong></td><td><strong>Error Description</strong></td><td><strong>Resolution</strong></td></tr><tr><td>1</td><td>Failed to establish a connection, Please check the credentials</td><td>Provide valid credentials or ensure proper access.</td></tr><tr><td>2</td><td>An error occurred while validating the DBT Core connection.</td><td><p><strong>403:</strong> Access denied [Provide appropriate access to user or role using in connection]</p><p><strong>404:</strong> No such key [The object does not exist in the remote.] </p></td></tr></tbody></table>

## **Manage Connector Operations**

### **Crawl/Profile**

{% hint style="warning" %}
***Important**: You must have the Integration Admin role in OvalEdge for crawl/profile operations.*
{% endhint %}

Crawl and Profile operations allow you to choose one or more schemas from a list of all available schemas within a specific database. This customization enables you to tailor the selection of crawling operations to meet your needs. Additionally, it offers the option to schedule crawls.

### **Other Operations**

The **Connectors page** in OvalEdge provides a centralized view of all configured connectors, including their health status. You can view, edit, validate, build lineage, and delete connectors using the **Nine Dots** menu.

#### Managing connectors includes:

* **Connectors Health**: Displays performance with a green (active) or red (inactive) icon, helping monitor data flow and address issues early.
* **Viewing**: Shows connector details (e.g., Databases, Tables, Table Columns, and Codes) via the **View** icon.

#### **Nine Dots Menu Options:**

* **Edit Connector**: Update and revalidate the data source.
* **Validate Connector**: Check the connection's integrity.
* **Settings**: Modify connector settings.
  * **Lineage:** The lineage settings are enabled by default, allowing you to select multiple sources for lineage building. Choose server dialects for parsing source codes, ensuring at least one is selected. Configure table priority to specify the order in which tables with the same name or signature are considered for lineage.
* **Build Lineage**: Automatically build data lineage using SQL logs and source code parsing.
* **Delete Connector**: Remove connectors or schemas with confirmation.   &#x20;

## **Additional Information**&#x20;

### **NFS Configuration**

* If the **DBT Core Repository** type is set to **NFS**, the **NFS path** is required.
* The **parent folder (dbt\_artifactory)** must contain only **DBT folders**.

### **S3 Configuration**

* The **S3 bucket name** must not include slashes (/ or \\). Only the bucket name is required.
* The **manifest.json** file must exist in the **target folder**.

***

Copyright © 2025, OvalEdge LLC, Peachtree Corners GA USA