# ETLs

This article provides a detailed overview of the enhanced Extract, Transform, Load (ETL) feature in OvalEdge. The ETL module enables users to configure, execute, and manage ETL pipelines that prepare data in staging tables for reporting and analytics.

ETL pipelines serve as the integration layer between OvalEdge and external reporting tools, such as Tableau and Power BI, by transforming OvalEdge metadata into structured reporting datasets.

This document explains the complete ETL workflow, including prerequisites, system settings, database setup, ETL creation, execution methods, configuration management, staging table visibility, and operational actions.

It also covers enhancements introduced in the ETL user interface, including centralized configuration pages, scheduling visibility, SQL script management, audit history, and simplified operational controls.

## Purpose 

The purpose of this document is to help users understand how to configure, execute, and manage ETL pipelines within OvalEdge. It provides step-by-step guidance for setting up the ETL environment, creating ETL jobs, monitoring schedules, managing staging tables, and maintaining ETL configurations.

## ETL Feature Overview

<table><thead><tr><th width="160">Component</th><th>Description</th></tr></thead><tbody><tr><td>Extraction</td><td>Retrieves required data from OvalEdge source tables.</td></tr><tr><td>Transformation</td><td>Applies SQL logic and business rules to prepare reporting data.</td></tr><tr><td>Loading</td><td>Loads processed data into staging tables for reporting and analytics.</td></tr><tr><td>Scheduling</td><td>Supports manual and scheduled ETL execution.</td></tr><tr><td>Governance</td><td>Provides audit visibility and controlled configuration management.</td></tr></tbody></table>

{% hint style="info" %}
The ETL feature is not accessible through the standard navigation. It must be accessed directly using the URL pattern: http\://\<your\_domain>/ovaledge/#nav/etl
{% endhint %}

## Prerequisites and Dependencies

<table><thead><tr><th width="210">Area</th><th>Details</th></tr></thead><tbody><tr><td>Access Rights</td><td>Users must have administrator-level access to configure ETL pipelines.</td></tr><tr><td>Database Connectivity</td><td>The OvalEdge instance must be connected to the target database where staging tables will be created.</td></tr><tr><td>Scheduling Permissions</td><td>Scheduling ETL runs requires that background jobs be configured on the server.</td></tr></tbody></table>

**For advanced reporting using external tools:**

1. Use the staging tables created by OvalEdge as a data source.
2. Connect external reporting tools such as Power BI or Tableau to the staging tables.
3. Build custom visualizations, metrics, and dashboards using these tools.
4. Use this option if reporting needs are complex or require advanced customization.

**How it works:**

* OvalEdge prepares the data and organizes it into staging tables.
* Access to these staging tables is provided.
* External tools connect directly to these tables.

## System Settings

Users must configure these System Settings to run ETL pipelines correctly by configuring the database, schema, and user credentials. This ensures ETL jobs run with proper permissions, maintain data consistency, and support external reporting tools.

1. Navigate to Administrator > **System Settings** > **Others**.
2. **Configure the following parameters:**
   1. etl.new\.connectorid: Enter the connector ID on which ETL jobs should be executed.
   2. etl.db.schema.name: Set the schema name for executing ETL jobs.
   3. etl.db.user.name: Specify the username with full access to the reporting DB and read access on OE DB.
   4. etl.db.user.password: Enter the password for the username.

      <figure><img src="/files/uNjenbY9gSg59EOHoTWw" alt=""><figcaption></figcaption></figure>

## Database Setup&#x20;

1. In the reporting environment, create a dedicated reporting database.
2. Create a dedicated user for database access and assign appropriate permissions.

{% hint style="info" %}
The user should have read access to the main database tables and views. Grant full privileges on the reporting database to allow reporting operations.
{% endhint %}

3. Create a new connector for the staging tables by navigating to Administrator > Connectors > Add New.

{% hint style="info" %}
This step is optional and can be skipped if the requirements are met.
{% endhint %}

4. Configure the connector's system settings to ensure proper access and functionality.

***

Copyright © 2026, OvalEdge LLC, Peachtree Corners, GA, USA.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.ovaledge.com/release8.1/advanced-tools/etls.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.