# AWS Glue ETL

An out-of-the-box connector is available for AWS Glue ETL entities. It provides support for crawling Jobs, Workflow, Triggers, Crawlers, and lineage building for the above entities.

## **Connectivity Summary**

<figure><img src="https://content.gitbook.com/content/hTnkoJQml0pok9awFDhx/blobs/fJUFZUrOaQ2XSqWJY6wh/AWS%20Glue%20ETL.webp" alt=""><figcaption></figcaption></figure>

The connectivity to AWS Glue ETL is via AWS Glue SDK, which is included in the platform.&#x20;

The Glue SDK used by the connector is given below:

<table data-header-hidden><thead><tr><th width="142.75"></th><th width="183.25"></th><th></th></tr></thead><tbody><tr><td><strong>Driver / API</strong></td><td><strong>Version</strong></td><td><strong>Details</strong></td></tr><tr><td>AWS Glue SDK</td><td>1.12.232</td><td><p><a href="https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-glue/1.12.232">https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-glue/1.12.232</a></p><p>Note: Latest version is 1.12.244.</p></td></tr></tbody></table>

## **Technical Specifications**

The connector capabilities are shown below:\
The AWS Glue entities are created as datasets as Jobs, Workflows, Crawlers, and Triggers. We extract a job's script and build the lineage for it whereas for the crawlers, triggers, and workflows we extract the information of entities involved and build the associations accordingly.

### **Crawling**

| Feature  | Supported Objects | Remarks |
| -------- | ----------------- | ------- |
| Crawling | Jobs              |         |
|          | Workflows         |         |
|          | Crawlers          |         |
|          | Triggers          |         |

### **Lineage Building**

| Lineage entities | Details     |
| ---------------- | ----------- |
| Jobs             | In Progress |
| Workflows        | Supported   |
| Crawlers         | Supported   |
| Triggers         | Supported   |

## **Pre-requisites:**

To use the connector, the following need to be available:

* Connection details as specified in the following section should be available.
* An admin/service account, for crawling. The minimum privileges required are

| **Operation**   | **Access Permission**             |
| --------------- | --------------------------------- |
| Crawl Jobs      | LIST, GET permission on Jobs      |
| Crawl Workflows | LIST, GET permission on workflows |
| Crawl Crawlers  | LIST, GET permission on crawlers  |
| Crawl Triggers  | LIST, GET permission on triggers  |

### **Connection Details**

The following connection settings should be added for connecting to AWS Glue ETL:

| Property        | Details                                                                                                                                                                                                   |
| --------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Database Type   | AWS Glue ETL                                                                                                                                                                                              |
| Connection Name | <p>Select a Connection name for the AWS Glue ETL. The name that you specify is a reference name to easily identify your AWS Glue ETL connection in OvalEdge. </p><p>Example: AWS Glue ETL Connection.</p> |
| Authentication  | Select the authentication type whether it is  Role-based authentication or Basic Authentication.                                                                                                          |
| Access key      | Access key                                                                                                                                                                                                |
| Secret key      | Secret key                                                                                                                                                                                                |
| Region          | Region of Glue                                                                                                                                                                                            |

## **Points to note:**

AWS Glue ETL doesn’t support querying for the Glue data catalog from OvalEdge.

***

Copyright © 2025, OvalEdge LLC, Peachtree Corners GA USA