ETLs

This article provides a detailed overview of the enhanced Extract, Transform, Load (ETL) feature in OvalEdge. The ETL module enables users to configure, execute, and manage ETL pipelines that prepare data in staging tables for reporting and analytics.

ETL pipelines serve as the integration layer between OvalEdge and external reporting tools, such as Tableau and Power BI, by transforming OvalEdge metadata into structured reporting datasets.

This document explains the complete ETL workflow, including prerequisites, system settings, database setup, ETL creation, execution methods, configuration management, staging table visibility, and operational actions.

It also covers enhancements introduced in the ETL user interface, including centralized configuration pages, scheduling visibility, SQL script management, audit history, and simplified operational controls.

Purpose

The purpose of this document is to help users understand how to configure, execute, and manage ETL pipelines within OvalEdge. It provides step-by-step guidance for setting up the ETL environment, creating ETL jobs, monitoring schedules, managing staging tables, and maintaining ETL configurations.

ETL Feature Overview

Component	Description
Extraction	Retrieves required data from OvalEdge source tables.
Transformation	Applies SQL logic and business rules to prepare reporting data.
Loading	Loads processed data into staging tables for reporting and analytics.
Scheduling	Supports manual and scheduled ETL execution.
Governance	Provides audit visibility and controlled configuration management.

The ETL feature is not accessible through the standard navigation. It must be accessed directly using the URL pattern: http://<your_domain>/ovaledge/#nav/etl

Prerequisites and Dependencies

Area	Details
Access Rights	Users must have administrator-level access to configure ETL pipelines.
Database Connectivity	The OvalEdge instance must be connected to the target database where staging tables will be created.
Scheduling Permissions	Scheduling ETL runs requires that background jobs be configured on the server.

For advanced reporting using external tools:

1. Use the staging tables created by OvalEdge as a data source.

2. Connect external reporting tools such as Power BI or Tableau to the staging tables.

3. Build custom visualizations, metrics, and dashboards using these tools.

4. Use this option if reporting needs are complex or require advanced customization.

How it works:

• OvalEdge prepares the data and organizes it into staging tables.

• Access to these staging tables is provided.

• External tools connect directly to these tables.

System Settings

Users must configure these System Settings to run ETL pipelines correctly by configuring the database, schema, and user credentials. This ensures ETL jobs run with proper permissions, maintain data consistency, and support external reporting tools.

1. Navigate to Administrator > System Settings > Others.

2. Configure the following parameters:

   1. etl.new.connectorid: Enter the connector ID on which ETL jobs should be executed.

   2. etl.db.schema.name: Set the schema name for executing ETL jobs.

   3. etl.db.user.name: Specify the username with full access to the reporting DB and read access on OE DB.

   4. etl.db.user.password: Enter the password for the username.

      

Database Setup

1. In the reporting environment, create a dedicated reporting database.

2. Create a dedicated user for database access and assign appropriate permissions.

The user should have read access to the main database tables and views. Grant full privileges on the reporting database to allow reporting operations.

1. Create a new connector for the staging tables by navigating to Administrator > Connectors > Add New.

This step is optional and can be skipped if the requirements are met.

1. Configure the connector's system settings to ensure proper access and functionality.

---

Copyright © 2026, OvalEdge LLC, Peachtree Corners, GA, USA.