> For the complete documentation index, see [llms.txt](https://docs.ovaledge.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.ovaledge.com/release8.1/connectors/connector-repositories/data-warehouse/amazon-redshift/amazon-redshift-lineage.md).

# Amazon Redshift - Lineage

This article outlines the lineage coverage, configuration settings, metadata handling, supported scenarios, component behaviors, process flow, and known limitations for lineage extraction in Amazon Redshift.

The Amazon Redshift connector provides automated lineage extraction from Redshift SQL artifacts, including tables, views, and stored procedures. It enables end-to-end visibility into data flow by parsing SQL queries and tracking upstream and downstream dependencies across data warehouse objects. The connector supports both table-level and column-level lineage, helping users understand data movement, perform impact analysis, troubleshoot dependencies, and support governance initiatives.

## Lineage Configuration Requirements

Accurate lineage extraction depends on specific configuration settings, access permissions, and SQL metadata availability. Configure the following settings correctly to ensure successful lineage extraction and relationship mapping.

### Configuration Requirements Table

<table><thead><tr><th width="219">Configuration Area</th><th>Required Detail</th></tr></thead><tbody><tr><td>Authentication</td><td>Configure valid Amazon Redshift connection credentials with access to the required schemas.</td></tr><tr><td>Access Requirements</td><td>Ensure access to database schemas, tables, and dataset SourceCode metadata. Valid SQL definitions for tables, views, and stored procedures are required for lineage extraction.</td></tr><tr><td>SQL Dialect Processing</td><td>Redshift-specific SQL parsing and cleanup logic are used to interpret SQL statements and build lineage relationships.</td></tr><tr><td>Processing Control</td><td>Configure lineage.codes.process.limit to control the volume of SQL code processed during lineage extraction.</td></tr><tr><td>Enhanced Parsing</td><td>Configure need.hard.coded.values.in.lineage to improve lineage coverage for hard-coded values and SQL expressions.</td></tr><tr><td>Self-Lineage Handling</td><td>Configure lineage.selfbuild to control self-lineage filtering and processing behavior.</td></tr></tbody></table>

{% hint style="info" %}
Using explicit table aliases, qualified column names, and properly formatted SQL statements can improve lineage accuracy and column-level mapping.
{% endhint %}

{% hint style="info" %}
Missing SourceCode metadata, inaccessible schemas, or invalid SQL definitions may prevent the connector from building complete lineage relationships.
{% endhint %}

## Lineage Components

| Component                               | Availability |
| --------------------------------------- | :----------: |
| Tables                                  |       ✅      |
| Views                                   |       ✅      |
| Stored Procedures                       |      ⚠️      |
| Functions (within SQL)                  |      ⚠️      |
| SQL Queries                             |       ✅      |
| Column Lineage                          |      ⚠️      |
| Temp Tables/Columns                     |       ✅      |
| Reports/Charts                          |       ❌      |
| Triggers                                |      ⚠️      |
| SELECT (standard queries)               |       ✅      |
| INSERT INTO SELECT                      |       ✅      |
| CREATE TABLE AS SELECT (CTAS)           |       ✅      |
| View Definition                         |       ✅      |
| MERGE / UPSERT                          |      ⚠️      |
| Stored Procedure SQL                    |      ⚠️      |
| Expressions                             |      ⚠️      |
| Aggregations                            |      ⚠️      |
| Dynamic SQL (EXEC)                      |       ❌      |
| OPENQUERY                               |       ❌      |
| Non-lineage DDL (INDEX, OPTIMIZE, etc.) |       ❌      |
| Direct Mapping                          |       ✅      |
| Renamed Columns                         |       ✅      |
| Derived Columns (with alias)            |      ⚠️      |
| Expressions                             |      ⚠️      |
| Aggregations                            |      ⚠️      |
| Temp Columns                            |       ✅      |
| Semantic Columns                        |      ⚠️      |

{% hint style="info" %}
The ⚠️ icon indicates partially supported functionality with limited lineage coverage in applicable scenarios.
{% endhint %}

## Supported Use Cases

The Amazon Redshift connector supports lineage extraction across SQL-based datasets, views, stored procedures, and transformation workflows. These scenarios represent the areas where lineage extraction functions as expected and provides reliable table-level and column-level lineage relationships.

### Supported Lineage Scenarios

<table><thead><tr><th width="220.33331298828125">Supported Scenario</th><th>Details</th></tr></thead><tbody><tr><td>Table-to-Table Lineage</td><td>Supports lineage extraction for CTAS (CREATE TABLE AS SELECT) and INSERT INTO ... SELECT operations to track data movement between source and target tables.</td></tr><tr><td>View Lineage</td><td>Extracts lineage from view definitions and maps relationships between views, source tables, and dependent views.</td></tr><tr><td>Column-Level Lineage</td><td>Supports source-to-target column mapping through SQL parsing, alias resolution, and column dependency analysis.</td></tr><tr><td>SQL-Based Transformations</td><td>Tracks lineage for SQL transformations that use explicit column references and supported SQL constructs.</td></tr><tr><td>Stored Procedure Lineage</td><td>Supports lineage extraction from stored procedures that contain static SQL statements.</td></tr><tr><td>Function-Level Lineage</td><td>Supports lineage extraction for functions used within SQL queries when source and target column relationships can be resolved.</td></tr><tr><td>Multi-Step SQL Transformations</td><td>Tracks lineage across multiple SQL operations and intermediate datasets to build end-to-end lineage relationships.</td></tr><tr><td>Temporary Object Handling</td><td>Supports lineage generation when temporary objects are used to resolve missing tables or columns during lineage processing.</td></tr></tbody></table>

{% hint style="info" %}
Column-level lineage accuracy improves when SQL statements use explicit column references, qualified column names, and clear alias definitions.
{% endhint %}

{% hint style="warning" %}
Stored procedure lineage is supported only for static SQL. Dynamic SQL and runtime-generated statements may result in incomplete lineage extraction.
{% endhint %}

## Partial or Limited Coverage

Certain lineage scenarios have partial coverage due to SQL complexity, metadata availability, or limitations in SQL parsing. These scenarios may produce incomplete lineage relationships or reduced column-level mapping accuracy.&#x20;

### Scenarios

<table><thead><tr><th width="219">Scenario</th><th>Limitation Description</th></tr></thead><tbody><tr><td>Stored Procedures</td><td>Lineage extraction is limited for complex stored procedures that contain mixed SQL patterns, nested logic, or unsupported constructs.</td></tr><tr><td>MERGE / UPSERT Statements</td><td>Column-level lineage mapping may be partial depending on statement complexity and transformation logic.</td></tr><tr><td>Ambiguous Columns</td><td>Unqualified column names may reduce the accuracy of column-level lineage and dependency resolution.</td></tr><tr><td>Derived Columns Without Aliases</td><td>Columns created without explicit aliases may result in incomplete source-to-target column mappings.</td></tr><tr><td>Function-Based Transformations</td><td>Lineage coverage depends on the ability to parse and resolve function logic within SQL statements.</td></tr><tr><td>Column Coordinate Mapping</td><td>Column coordinate visualization in the lineage UI depends on the availability of required configuration and metadata.</td></tr><tr><td>Temporary Tables and Columns</td><td>When metadata is unavailable, temporary tables and columns may be created with an UNKNOWN type, resulting in partial lineage.</td></tr></tbody></table>

{% hint style="info" %}
Using qualified column names, explicit column aliases, and well-structured SQL statements can improve lineage accuracy and column-level mapping.
{% endhint %}

{% hint style="warning" %}
Missing SourceCode metadata, complex stored procedure logic, unsupported SQL constructs, or dynamically generated SQL may prevent the connector from building complete lineage relationships.
{% endhint %}

## Unsupported Scenarios

The Amazon Redshift connector does not support lineage extraction for certain dynamic execution patterns, administrative operations, external query mechanisms, and non-SQL assets due to limitations in SQL parsing and metadata availability.

### Unsupported Lineage

<table><thead><tr><th width="219.66668701171875">Not Supported</th><th>Description</th></tr></thead><tbody><tr><td>Dynamic SQL</td><td>Runtime-generated SQL statements, including EXEC-based execution patterns, are not supported for lineage extraction.</td></tr><tr><td>OPENQUERY and External Query Execution</td><td>Queries executed through external query mechanisms are not processed for lineage generation.</td></tr><tr><td>Non-Lineage DDL Operations</td><td>Administrative operations such as CREATE INDEX, DROP INDEX, ALTER INDEX, OPTIMIZE, and DESC statements are not included in lineage extraction.</td></tr><tr><td>REFRESH MATERIALIZED VIEW</td><td>Materialized view refresh operations are not tracked as lineage relationships.</td></tr><tr><td>Unsupported SQL Syntax</td><td>Queries containing unsupported or unrecognized SQL syntax patterns may be excluded from lineage processing.</td></tr><tr><td>ETL Package Formats</td><td>External ETL package formats, such as .dtsx files, are outside the scope of the Amazon Redshift connector.</td></tr><tr><td>Complex Procedural Logic</td><td>Advanced procedural logic that cannot be resolved through SQL parsing is not supported for lineage extraction.</td></tr></tbody></table>

{% hint style="info" %}
Unsupported scenarios do not generate lineage relationships and may appear disconnected or absent in lineage visualizations.
{% endhint %}

{% hint style="warning" %}
For lineage involving ETL tools, reporting platforms, or external processing frameworks, use the corresponding connector that supports those assets and metadata sources.
{% endhint %}

## Current Functional Status

This section describes the current lineage coverage and functional behavior supported by the Amazon Redshift connector.

<table><thead><tr><th width="219.66668701171875">Status Area</th><th>Details</th></tr></thead><tbody><tr><td>Overall Coverage</td><td>Partial coverage across Amazon Redshift tables, views, stored procedures, and SQL-based transformation workflows.</td></tr><tr><td>Lineage Depth</td><td>Supports reliable table-level lineage and partial column-level lineage through SQL parsing and dependency analysis.</td></tr><tr><td>Supported Inputs</td><td>Tables, views, stored procedures, SQL datasets, and supported SQL transformation statements.</td></tr><tr><td>Functional Scope</td><td>Lineage extraction works effectively for standard SQL operations, including CTAS, INSERT INTO ... SELECT, view definitions, and supported SQL transformations.</td></tr><tr><td>Limitation Areas</td><td>Dynamic SQL, complex stored procedures, ambiguous column references, unsupported SQL syntax patterns, and certain advanced transformation scenarios.</td></tr><tr><td>Resulting Output</td><td>Produces reliable lineage for most Redshift SQL artifacts and transformation workflows, with partial mapping in complex or unsupported scenarios.</td></tr></tbody></table>

{% hint style="info" %}
The Amazon Redshift connector provides strong lineage capabilities for tables, views, and SQL-based transformations, including column-level lineage for supported SQL patterns. However, dynamic SQL, complex stored procedures, and ambiguous column mappings may result in incomplete lineage and could require manual validation.
{% endhint %}

***

&#x20;Copyright © 2026, OvalEdge LLC, Peachtree Corners GA USA


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.ovaledge.com/release8.1/connectors/connector-repositories/data-warehouse/amazon-redshift/amazon-redshift-lineage.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.