Dremio

This article outlines the integration with the Dremio connector, enabling streamlined metadata management through features such as crawling, profiling, querying, data preview, data quality, and lineage building (both automatic and manual).

This connector supports connectivity to Dremio environments using Traditional Jar and Arrow Flight JDBC drivers, providing flexibility based on deployment and performance requirements. It supports metadata extraction for schemas, tables, views, and columns.

Overview

Connector Details

Connector Category	RDBMS
OvalEdge Release Supported	Release6.3.4 and later
Connectivity [How the connection is established with Dremio]	JDBC driver
Verified Dremio Version	Dremio 26.x

The Dremio connector has been validated with the mentioned "Verified Dremio Version" and is expected to be compatible with other supported Dremio versions. If there are any issues with validation or metadata crawling, please submit a support ticket for investigation and feedback.

Connector Features

Feature	Availability
Crawling	✅
Delta Crawling	❌
Profiling	✅
Sample Profiling	✅
Query Sheet	✅
Data Preview	✅
Auto Lineage	✅
Manual Lineage	✅
Secure Authentication via Credential Manager	✅
Data Quality	✅
DAM (Data Access Management)	❌
Bridge	✅

Metadata Mapping

The following objects are crawled from Dremio and mapped to the corresponding UI assets.

Dremio Object	Dremio Attribute	OvalEdge Attribute	OvalEdge Category	OvalEdge Type
Schema	SCHEMA_NAME	Schema	Schemas	-
Table	TABLE_NAME	Table	Tables	Table
TABLE_TYPE	Table	Tables	Table	-
Columns	COLUMN_NAME	Column	Table Columns	-
DATA_TYPE	Column Type	Table Columns	-	-
ORDINAL_POSITION	Column Position	Table Columns	-	-
COLUMN_SIZE	Data Type Size	Table Columns	-	-
Views	TABLE_NAME	View Name	Tables	View
VIEW_DEFINITION	View Query	Views	View	-

Set up a Connection

Prerequisites

The following are the prerequisites to establish a connection:

Whitelisting Ports

Ensure that the required inbound ports are whitelisted to allow OvalEdge to establish a connection with the Dremio server. For connections using the Traditional JDBC Driver, the default port 31010 must be accessible, while connections using the Arrow Flight JDBC Driver default port 32010 to be accessible.

If custom ports are configured in Dremio, ensure that the updated port numbers are specified during connection setup, the corresponding ports are whitelisted, and communication between the OvalEdge application and Dremio server is properly established.

External Supporting Files

The required external JAR files are included as part of the OvalEdge installation artifacts. For driver installation and configuration details, refer to the Connector Drivers Setup Guide. Please contact the OvalEdge Team for assistance related to the driver files and configuration setup.

File Name	Description
flight-sql-jdbc-driver-10.0.0.jar	For Dremio versions earlier than 21.x, use the JDBC Driver version 10.0.0 to establish connectivity.

Service Account User Permissions

It is recommended to use a separate service account to establish the connection to the data source, configured with the following minimum set of permissions.

👨‍💻 Who can provide these permissions? These permissions are typically granted by the Dremio administrator, as users may not have the required access to assign them independently.

Operation	Objects	System Tables	Access Permission
Crawling	Schema	INFORMATION_SCHEMA.SCHEMATA	SELECT
Crawling	Tables	INFORMATION_SCHEMA.TABLES	SELECT
Crawling	Table Columns	INFORMATION_SCHEMA.COLUMNS	SELECT
Crawling & Lineage Building	Views	INFORMATION_SCHEMA.VIEWS	SELECT
Profiling	Row Count	User Data Tables	SELECT
Profiling	Data Profiling	User Data Tables	SELECT
Profiling	Sample Data	User Data Tables	SELECT
Profiling	Column Statistics	User Data Tables	SELECT
Crawling	Column Matching	User Data Tables	SELECT
Query Logging	Query Logs	NAS.queries.json	SELECT
Data Access & Governance	Governed Data Query Execution	User Data Tables	SELECT
Data Access & Query Execution	Data Query Execution (Async)	User Data Tables	SELECT, INSERT, UPDATE, DELETE, CREATE, ALTER, DROP
Data Access & Query Execution	Data Query Execution (Real-Time)	User Data Tables	SELECT, INSERT, UPDATE, DELETE, CREATE, ALTER, DROP
Connection Validation	-	INFORMATION_SCHEMA.VIEWS	SELECT

Connection Configuration Steps

Users are required to have the Connector Creator role in order to configure a new connection.

1. Log into OvalEdge, go to Administration > Connectors, click + (New Connector), search for Dremio, and complete the required parameters.

Fields marked with an asterisk (*) are mandatory for establishing a connection.

Field Name	Description
Connector Type	By default, "Dremio" is displayed as the selected connector type.
Credential Manager*	Select the desired credentials manager from the drop-down list. Relevant parameters will be displayed based on your selection. Supported Credential Managers: OE Credential Manager AWS Secrets Manager HashiCorp Vault Azure Key Vault For more details, click here.
License Add Ons	Select the checkbox for Auto Lineage Add-On to build data lineage automatically. Select the checkbox for Data Quality Add-On to identify data quality issues using data quality rules and anomaly detection. For more details, click here.
Connector Name*	Enter a unique name for the Dremio connection (Example: "Dremio").
Connector Environment	Select the environment (Example: PROD, STG) configured for the connector. For more details, click here.
JDBC Driver*	Select the JDBC driver type to connect to Dremio: Traditional Jar – Uses the standard Dremio JDBC connection. Arrow Flight – Uses Apache Arrow Flight Dremio for high-performance data transfer.
Connector description	Enter a brief description of the connector.
Server*	Enter the Dremio database server name or IP address Example: dremio.*****ny.com 192.168.1.100
Port*	By default, the port number for the Dremio, "31010" is auto-populated for Traditional Jar. If required, the port number can be modified as per the custom port number that is configured for your Dremio. Note: If you are using the Arrow Flight JDBC Driver, ensure that the inbound port 32010 by default is also whitelisted.
Disable Certificate Verification*	Select True to bypass SSL certificate validation during connection. Select False to enforce certificate verification for secure communication.
Is SSL Enabled*	Select True to enable SSL/TLS encryption for the connection. Select False to connect without SSL encryption.
Driver*	By default, the Dremio driver details are auto-populated.
Username*	Enter the service account username set up to access the Dremio database (Example: "oesauser").
Password*	Enter the password associated with the service account user.
Connection String	Configure the connection string for the Dremio database: Automatic Mode: The system generates a connection string based on the provided credentials. Manual Mode: Enter a valid connection string manually. Replace placeholders with actual database details. {sid} refers to Database Name.
Plugin Server	Enter the server’s name when running as a plugin server.
Plugin Port	Enter the port number on which the plugin is running.

Default Governance Roles
Default Governance Roles*	Select the appropriate users or teams for each governance role from the drop-down list. All users and teams configured in OvalEdge Security are displayed for selection.
Admin Roles
Admin Roles*	Select one or more users from the dropdown list for Integration Admin and Security & Governance Admin. All users configured in OvalEdge Security are available for selection.
No of Archive Objects
No Of Archive Objects*	This shows the number of recent metadata changes to a dataset at the source. By default, it is off. To enable it, toggle the Archive button and specify the number of objects to archive. Example: Setting it to 4 retrieves the last four changes, displayed in the 'Version' column of the 'Metadata Changes' module.
Bridge
Select Bridge*	If applicable, select the bridge from the drop-down list. The drop-down list displays all active bridges configured in OvalEdge. These bridges enable communication between data sources and OvalEdge without altering firewall rules.

1. After entering all connection details, the following actions can be performed:

   • Click Validate to verify the connection.

   • Click Save to store the connection for future use.

   • Click Save & Configure to apply additional settings before saving.

2. The saved connection will appear on the Connectors home page.

Manage Connector Operations

Crawl/Profile

To perform crawl and profile operations, users must be assigned the Integration Admin role.

The Crawl/Profile button allows users to select one or more schemas for crawling and profiling.

1. Navigate to the Connectors page and click Crawl/Profile.

2. Select the schemas to crawl.

3. The Crawl option is selected by default. Click the Crawl & Profile radio button to run both operations.

4. Click Run to collect metadata from the connected source and load it into the OvalEdge Data Catalog.

5. After a successful crawl, the information appears in the Data Catalog > Databases tab.

The Schedule checkbox allows automated crawling and profiling at defined intervals, from a minute to a year.

1. Click the Schedule checkbox to enable the Select Period drop-down.

2. Select a time period for the operation from the drop-down menu.

3. Click Schedule to initiate metadata collection from the connected source.

4. The system will automatically execute the selected operation (Crawl or Crawl & Profile) at the scheduled time.

Other Operations

The Connectors page in the OvalEdge provides a centralized view of all configured connectors, including their health status.

Managing connectors includes:

• Connectors Health: Displays the current status of each connector using a green icon for active connections and a red icon for inactive connections, helping to monitor the connectivity with data sources.

• Viewing: Click the Eye icon next to the connector name to view connector details.

Nine Dots Menu Options:

To view, edit, validate, build lineage, configure, or delete connectors, click on the Nine Dots menu.

• Edit Connector: Update and revalidate the data source.

• Validate Connector: Check the connection's integrity.

• Settings: Modify connector settings.

  • Crawler: Configure data extraction.

  • Profiler: Customize data profiling rules and methods.

  • Access Instructions: Add notes on how data can be accessed.

  • Query Policies: Define query execution rules based on roles.

  • Business Glossary Settings: Manage term associations at the connector level.

  • Anomaly Detection Settings: Configure anomaly detection preferences at the connector level.

  • Others: Configure notification recipients for metadata changes.

• Build Lineage: Automatically build data lineage from source code.

• Delete Connector: Remove a connector with confirmation.

For more details on connector settings, click here.

Connectivity Troubleshooting

If incorrect parameters are entered, error messages may appear. Ensure all inputs are accurate to resolve these issues. If issues persist, contact the assigned support team.

S.No.	Error Message(s)	Error Description & Resolution
1	Connection refused Connection timeout	Error Description: The OvalEdge application cannot establish a network connection to the Dremio server. Resolution: Verify that the server hostname or IP address is correct. Ensure the correct port is configured (Arrow Flight: 32010, Traditional JDBC: 31010). Confirm that the Dremio server is running and reachable. Verify that firewall rules allow communication through the configured port. Test connectivity using telnet <server> <port>. Ensure the connection string matches the selected driver type. Verify SSL settings if SSL is enabled.
2	Failed to retrieve DataSource from DataSourceProvider	Error Description: The connection pool could not initialize the Dremio data source. Resolution: Verify the correct JDBC driver class is configured: Ensure the JDBC driver JAR is available in the OvalEdge classpath. Verify the connection string format. Confirm network and SSL settings are valid. Recreate the connection if cached connection details are causing issues. Verify that the driver version is compatible with the Dremio server version.
3	SSL connection failed Certificate validation error	Error Description: The SSL connection could not be established due to certificate validation issues or incorrect SSL configuration. Resolution: Verify that Is SSL Enabled matches the Dremio server configuration. Configure Disable Certificate Verification appropriately. Ensure SSL certificates are valid and trusted. Verify the SSL parameters in the JDBC connection string. If using self-signed certificates, enable certificate verification bypass only if permitted by your security policies.
4	validateConnection() failed	Error Description: Connection validation failed even though the connection credentials are correct. Resolution: Verify that the user has access to the INFORMATION_SCHEMA views. Ensure the JDBC driver is installed and loaded correctly. Confirm that network connectivity and SSL settings are working. Verify that the validation query can be executed directly in Dremio. Review OvalEdge and Dremio logs for detailed error information.
5	No schemas returned getRemoteDatabases() failed	Error Description: OvalEdge cannot retrieve schema information from Dremio. Resolution: Verify that the user has access to INFORMATION_SCHEMA.SCHEMATA. Ensure the Dremio environment contains accessible schemas. Confirm that the service account has sufficient permissions. Review logs for SQL or permission-related errors. Validate schema access directly from Dremio.
6	Unable to connect after changing JDBC driver type	Error Description: The configured driver type does not match the Dremio server version or connection settings. Resolution: Use Arrow Flight JDBC Driver for Dremio version 21.x and later. Use Traditional JDBC Driver for older Dremio versions if required. Verify that the corresponding port is configured correctly. Ensure the connection string format matches the selected driver type. Recreate the connection if the driver type was changed after setup.
7	Connection failed due to incorrect port configuration	Error Description: The configured port does not match the Dremio service endpoint. Resolution: Arrow Flight JDBC Driver: 32010. Traditional JDBC Driver: 31010. Verify that any custom port configured in Dremio is also configured in OvalEdge. Test connectivity using telnet or another network utility. Ensure firewall rules allow access to the configured port.

---

Copyright © 2026, OvalEdge LLC, Peachtree Corners GA USA