# ECS-Variant-2: New Customers

This document outlines the end-to-end process for enabling AskEdgi components for newly onboarded SaaS customers on ECS-Variant-2. The implementation leverages AWS Control Tower automation to create and manage core AWS resources, while incorporating manual verification steps to ensure correct configuration and functionality.

AskEdgi components rely on a combination of AWS services such as ECS, RDS, EFS, S3, SQS, IAM, and Secrets Manager. These components are configured to interact seamlessly, enabling secure and scalable operations for the SaaS environment. This document provides a structured, step-by-step guide for setting up network security, IAM roles, ECS tasks, S3 buckets, SQS queues, event notifications, Bridge server and client setup, and feature validation for AskEdgi.

<div align="left"><figure><img src="/files/IG2158CkoBRGoudFGLaN" alt=""><figcaption></figcaption></figure></div>

<div align="left"><figure><img src="/files/xw9R6IFtC7Je58sMn6V3" alt=""><figcaption></figcaption></figure></div>

## Purpose of the document

The purpose of this document is to provide a detailed, professional, and structured process for implementing AskEdgi components on ECS-Variant-2 for new SaaS customers. It ensures all AWS resources, ECS tasks, S3 buckets, SQS queues, IAM configurations, and application services are configured correctly. It also provides verification steps to validate operational readiness and integration of AskEdgi within the Ovaledge environment.

## Prerequisites

<table><thead><tr><th width="134.45452880859375">Component</th><th width="293.272705078125">Requirement</th><th>Notes / Important</th></tr></thead><tbody><tr><td>ECS</td><td>ECS cluster and service access</td><td>Required for deploying Edgi Data and Insight services</td></tr><tr><td>SQS</td><td>Access to create and manage queues</td><td>Needed for data sync between services</td></tr><tr><td>IAM</td><td>Permissions to create roles, users, and attach policies</td><td>Ensure IAM policies include required permissions for ECS, SQS, and S3</td></tr><tr><td>RDS</td><td>Database instance</td><td>Used for storing Edgi data and metadata</td></tr><tr><td>EFS</td><td>File system access</td><td>Required for persistent storage for the Insight service</td></tr><tr><td>Secrets Manager</td><td>Create and manage secrets</td><td>Used to store database credentials and other sensitive keys</td></tr></tbody></table>

{% hint style="warning" %}

* Ensure AWS Control Tower automation is active and operational before starting.
* Manual verification is required for ECS task roles, IAM user credentials, S3 bucket permissions, and event notifications.
* All services should comply with corporate security policies regarding VPCs, subnets, and access controls.
  {% endhint %}

## Implementation Steps

### Step 1: Network Security Configuration

* Create two private subnets for the Edgi-Data.
* Create a Security Group for the Edgi-Data ECS service with the following rules:

<table><thead><tr><th width="157.18182373046875">Protocol</th><th width="176.8182373046875">Port</th><th>Source/Destination</th></tr></thead><tbody><tr><td>TCP</td><td>8000</td><td>Destination VPC range</td></tr><tr><td>TCP</td><td>9200</td><td>Private Elasticsearch Security Group</td></tr><tr><td>TCP</td><td>3306</td><td>Private RDS Security Group</td></tr><tr><td>All</td><td>All</td><td>Private EFS Security Group</td></tr></tbody></table>

{% hint style="info" %}

* Ensure that ports 8000, 9200, and 3306 are allowed only from specified sources.
* Security Group configuration ensures proper communication between Edgi ECS, RDS, EFS, and Elasticsearch.
  {% endhint %}

### Step 2: EFS Access Points

* Create EFS Access Points for askEdgi Insight Service and one common for Ovaledge UI/Job, data task definition, and insight.

| Access Point Path        | Notes         |
| ------------------------ | ------------- |
| /oe\_edgi\_assets\_store | Stores assets |

{% hint style="warning" %}
Assign correct mount permissions to ECS tasks to ensure persistent storage access.
{% endhint %}

### Step 3: S3 Bucket Creation

* Create a new S3 bucket for storing Edgi-related data.<br>

  <figure><img src="/files/sHvxLlGTHIAVpqKjKjXn" alt=""><figcaption></figcaption></figure>
* Configure CORS permissions in the bucket settings as shown below:

```
[
  {
    "AllowedHeaders": ["*"],
    "AllowedMethods": ["PUT"],
    "AllowedOrigins": ["*"],
    "ExposeHeaders": []
  }
]
```

<figure><img src="/files/l7qXbRbmWLro8iEa5tnG" alt=""><figcaption></figcaption></figure>

{% hint style="info" %}

* The bucket will store all data uploaded via Edgi Data Service and processed by the Insight Service.
* Ensure correct permissions to allow S3 to send events to SQS.
  {% endhint %}

### Step 4: IAM User Configuration

**IAM User for AskEdgi Python**

* Create IAM user (example: edgi\_user) with:

  * AskEdgi\_policy\_Saas
  * AmazonSQSFullAccess: <https://oe-saas-distributions-bridgeclient.s3.ap-south-1.amazonaws.com/Askedgi/AWS_SAAS_ECS_Policy.json>

  <figure><img src="/files/4l35tSTPXE7G7Zmeas6t" alt=""><figcaption></figcaption></figure>

  <figure><img src="/files/VGdtIgkgSgh6BsaU6e07" alt=""><figcaption></figcaption></figure>
* Generate Access Key and Secret Key for CLI access.<br>

  <figure><img src="/files/lLe05sDkudrgtOJLfpYw" alt=""><figcaption></figcaption></figure>
* Use credentials in Ovaledge Task Definition environment variables:

| Environment Variable   | Value         |
| ---------------------- | ------------- |
| EDGI\_AWS\_ACCESS\_ID  | \<access key> |
| EDGI\_AWS\_SECRET\_KEY | \<secret key> |

#### IAM User for Bridge Server and Client

* Create another IAM user with S3 PutObject permissions only.

<figure><img src="/files/GlW3GZ07O5Oxxq2P5j46" alt=""><figcaption></figcaption></figure>

* Use credentials in Ovaledge Task Definition environment variables:

<table><thead><tr><th width="374">Environment Variable</th><th>Value</th></tr></thead><tbody><tr><td>BRIDGE_AWS_USER_ACCESS_KEY</td><td>&#x3C;bridge IAM access key></td></tr><tr><td>BRIDGE_AWS_USER_SECRET_KEY</td><td>&#x3C;bridge IAM secret key></td></tr></tbody></table>

{% hint style="info" %}

* Bridge user credentials are required for secure S3 uploads.
* Validate IAM policies before ECS task deployment.
  {% endhint %}

### Step 5: ECS Task Role Configuration

* Create an IAM Role with ECS task execution permissions.
* Attach the custom policy: [AskEdgi ECS Policy JSON.](https://oe-saas-distributions-bridgeclient.s3.ap-south-1.amazonaws.com/Askedgi/AWS_SAAS_ECS_Policy.json)<br>

  <figure><img src="/files/uYDu0cZvWvROCEDhIY0H" alt=""><figcaption></figcaption></figure>
* Modify the Trust Relationship for ECS.<br>

  <figure><img src="/files/BsXwXLQFA6H5ZwlI708h" alt=""><figcaption></figcaption></figure>

{% hint style="warning" %}

* This IAM role is shared across all ECS Task Definitions to ensure consistent and secure access.
* Automation scripts handle permission setup automatically, but verification is recommended.
  {% endhint %}

### Step 6: S3 Event Notifications

Amazon S3 Event Notifications allow automatic triggering of actions when certain events happen in an S3 bucket - like when an object is uploaded, deleted, or restored.

To enable automated processing of uploaded files, configure S3 Event Notifications to trigger the SQS queue:

* Navigate to the S3 bucket > Properties > Event Notifications > Click Create Event Notification.
* Create Notification for CSV Files
  * Name: csv\_event\_to\_sqs (customizable)
  * File Suffix: .csv
  * Event Type: All object creation events
  * Destination: SQS
  * SQS ARN: {PREFIX}\_data\_sync\_queue

<figure><img src="/files/Qum1HFpmUEI9JHd8gzz2" alt=""><figcaption></figcaption></figure>

<figure><img src="/files/JPDmqaCejN8rjpiFhwEZ" alt=""><figcaption></figcaption></figure>

<figure><img src="/files/4B9OJ6fqfWsBEAY0YLZB" alt=""><figcaption></figcaption></figure>

<figure><img src="/files/X1SUqnUqcgVDkizBn0Sc" alt=""><figcaption></figcaption></figure>

* **Create Additional Notifications**\
  Repeat the above steps for the following file types, ensuring each notification points to the same SQS queue:

<table><thead><tr><th width="167.90911865234375">Notification Name</th><th width="119.3636474609375">File Suffix</th><th>Event Type</th><th>Destination</th></tr></thead><tbody><tr><td>avro_event</td><td>.avro</td><td>All object creation events</td><td>data_sync_queue</td></tr><tr><td>json_event</td><td>.json</td><td>All object creation events</td><td>data_sync_queue</td></tr><tr><td>parquet_event</td><td>.parquet</td><td>All object creation events</td><td>data_sync_queue</td></tr><tr><td>xlsx_event</td><td>.xlsx</td><td>All object creation events</td><td>data_sync_queue</td></tr></tbody></table>

<figure><img src="/files/G4EVkXQTra1fyUXKgDPw" alt=""><figcaption></figcaption></figure>

{% hint style="info" %}

* These notifications enable real-time processing of files uploaded to the S3 bucket.
* Verify that the correct SQS queue ARN is used for each notification.
  {% endhint %}

### Step 7: ECS Task Definition – Data Service

Create an ECS Task Definition for the Edgi Data Service with the following configuration:

| Configuration   | Value             |
| --------------- | ----------------- |
| CPU             | 0.5 vCPU          |
| Memory          | 2 GB              |
| Container Name  | edgi-data-service |
| Port            | 8000              |
| Storage         | 200 GB            |
| CloudWatch Logs | Enabled           |

#### **Environment Variables:**

| Key               | Value                   |
| ----------------- | ----------------------- |
| AWS\_REGION       | \<Region>               |
| AWS\_SCTNAME      | \<Database secret name> |
| BUCKET\_NAME      | \<Bucket Name>          |
| CREDENTIAL\_TYPE  | container               |
| DEPLOYMENT\_TYPE  | ecs                     |
| ENV               | production              |
| EDGI\_ASSET\_PATH | /app/asset              |
| OVALEDGE\_URL     | \<OE\_Application\_URL> |
| LOG\_LEVEL        | debug                   |

<figure><img src="/files/nOkISm2eY4aDDphWH6iu" alt=""><figcaption></figcaption></figure>

**Additional Configuration**

* Enable CloudWatch Logs for the container.
* Allocate a maximum of 200 GB under the storage configuration.

<figure><img src="/files/sYRNKprSXfR1iRD5JNNh" alt=""><figcaption></figcaption></figure>

{% hint style="warning" %}
CloudWatch logging is essential for monitoring Data Service activity.
{% endhint %}

* **Elastic File System Configuration**
  * Add EFS mounts for the data task definition.

<figure><img src="/files/gALMSNqQLMiSyfXZXqUe" alt=""><figcaption></figcaption></figure>

### Step 8: ECS Task Definition - Data Service Medium

* Create an ECS Task Definition for the Edgi Data Service Medium with the following configuration details.

| Configuration   | Value             |
| --------------- | ----------------- |
| CPU             | 1 vCPU            |
| Memory          | 8 GB              |
| Container Name  | edgi-data-service |
| Port            | 8000              |
| Storage         | 200 GB            |
| CloudWatch Logs | Enabled           |

**Environment Variables**

| Key               | Value                   |
| ----------------- | ----------------------- |
| AWS\_REGION       | \<Region>               |
| AWS\_SCTNAME      | \<Database secret name> |
| BUCKET\_NAME      | \<Bucket name>          |
| CREDENTIAL\_TYPE  | container               |
| DEPLOYMENT\_TYPE  | ecs                     |
| ENV               | production              |
| EDGI\_ASSET\_PATH | /app/asset              |
| OVALEDGE\_URL     | \<OE\_Application\_URL  |
| LOG\_LEVEL        | debug                   |

<figure><img src="/files/iTKcT5uZztaePKAZe4di" alt=""><figcaption></figcaption></figure>

**Additional Configuration**

* Enable CloudWatch Logs for the container to capture logs.
* Under Storage configuration, allocate a maximum of 200 GB.

<figure><img src="/files/n4QlOxLn0IPh9LAzEAW9" alt=""><figcaption></figcaption></figure>

{% hint style="warning" %}
CloudWatch logging is essential for monitoring Data Service activity.
{% endhint %}

**Elastic File System Configuration**

* Add EFS mounts for the data task definition.

<figure><img src="/files/JoKRpmOYjesnfrQi3bu9" alt=""><figcaption></figcaption></figure>

### Step 9: ECS Task Definition – Data Service-XL

Create an ECS Task Definition for the Edgi Data Service-XL with the following configuration details.

| Configuration   | Value             |
| --------------- | ----------------- |
| CPU             | 2 vCPU            |
| Memory          | 16 GB             |
| Container Name  | edgi-data-service |
| Port            | 8000              |
| Storage         | 200 GB            |
| CloudWatch Logs | Enabled           |

#### Environment Variables

| Key               | Value                   |
| ----------------- | ----------------------- |
| AWS\_REGION       | \<Region>               |
| AWS\_SCTNAME      | \<Database secret name> |
| BUCKET\_NAME      | \<Bucket Name>          |
| CREDENTIAL\_TYPE  | container               |
| DEPLOYMENT\_TYPE  | ecs                     |
| ENV               | production              |
| EDGI\_ASSET\_PATH | /app/asset              |
| OVALEDGE\_URL     | \<OE\_Application\_URL  |
| LOG\_LEVEL        | debug                   |

<figure><img src="/files/NYxq4tVbq8C5fxujQRm0" alt=""><figcaption></figcaption></figure>

* Enable CloudWatch Logs for the container to capture logs.
* Under Storage configuration, allocate a maximum of 200 GB.

<figure><img src="/files/sPrMbG47memvFVVQ6mYD" alt=""><figcaption></figcaption></figure>

{% hint style="info" %}
CloudWatch logging is essential for monitoring Data Service activity.
{% endhint %}

**Elastic File System Configuration**

* Add EFS mounts for the data task definition.

<figure><img src="/files/EViw7cYdGki59m3oXeHR" alt=""><figcaption></figcaption></figure>

{% hint style="info" %}
Service Discovery endpoint is required in Ovaledge task definitions.
{% endhint %}

### Step 10: API Key set in the AWS Secret Manager&#x20;

In the 8.1 Feature, an API Key update is introduced in the AWS Secret Manager.

The user needs to add a key without a value in the Secret Manager, which you are using for the DB details secret.

**apiaccesskeyforedgi**

<figure><img src="/files/VyvjNBBxOOo8XmPyevPO" alt=""><figcaption></figcaption></figure>

Once the application is started, the value will be stored in the secret manager with encryption.

**Permissions**: Provide the Update permission to the IAM role.

After the application is up and running, the API key will be added to the secret manager.

<figure><img src="/files/wPwMs2DUicXxl1IKTvjy" alt=""><figcaption></figcaption></figure>

### Step 11: Update Ovaledge Task Definition (UI and Job)

Update both the UI and Job ECS Task Definitions with the following environment variables to enable AskEdgi functionality:

**Environment Variables for UI and Job Tasks**

| Key                                  | Value                                             |
| ------------------------------------ | ------------------------------------------------- |
| BRIDGE\_AWS\_USER\_ACCESS\_KEY       | \<Bridge user access key>                         |
| BRIDGE\_AWS\_USER\_SECRET\_KEY       | \<Bridge user secret key>                         |
| EDGI\_AI\_FUNCTION\_QUEUE            | ai\_functions\_queue                              |
| EDGI\_AWS\_ACCESS\_ID                | \<edgi user access key>                           |
| EDGI\_AWS\_SECRET\_KEY               | \<edgi user secret key>                           |
| EDGI\_AWS\_REGION                    | \<region>                                         |
| EDGI\_BUCKET\_NAME                   | \<s3 bucket name>                                 |
| EDGI\_CLIENT\_PREFIX                 | \<sqs prefix name>                                |
| EDGI\_CONSUMER\_CREDENTIAL\_TYPE     | static                                            |
| EDGI\_ENABLE                         | true                                              |
| EDGI\_INSIGHT\_SERVICE\_HOST         | \<insight service endpoint>                       |
| EDGI\_WORKSPACE\_DISCOVERY           | dynamic                                           |
| EDGI\_WORKSPACE\_HOST                | \<edgi data task definition family name>          |
| EDGI\_WORKSPACE\_NAMESPACE           | \<ECS cluster namespace>                          |
| EDGI\_ASSET\_PATH                    | /app/asset                                        |
| EDGI\_WORKSPACE\_NAMESPACE\_ID       | \<ECS cluster namespace ID>                       |
| EDGI\_CLUSTER\_NAME                  | \<ECS Cluster name>                               |
| EDGI\_SUBNETS                        | \<ECS Private subsets az1,az2>                    |
| EDGI\_SECURITY\_GROUP                | \<secuirty group>                                 |
| EDGI\_STANDARD\_DATATASK\_FAMILYNAME | < data-service task definition family name        |
| EDGI\_MEDIUM\_DATATASK\_FAMILYNAME   | < data-service-medium task definition family name |
| EDGI\_LARGE\_DATATASK\_FAMILYNAME    | < data-service-xl task definition family name     |

**Optional OAUTH2 Configuration**

| Key                          | Value                            |
| ---------------------------- | -------------------------------- |
| OAUTH2\_BASE\_URL            | \<dev team will provide the URL> |
| OAUTH2\_CLIENTID             | \<dev team will provide the URL> |
| OAUTH2\_CLIENT\_REGISTRATION | auth0                            |
| OAUTH2\_ROLE\_ATTRIBUTE      | https\://\<Domain\_name>/roles   |
| OAUTH2\_SECRET               | \<dev team will provide the URL> |
| OVALEDGE\_SECURITY\_TYPE     | oauth2                           |

<figure><img src="/files/XM70WowH6xGd6KJtkOif" alt=""><figcaption></figcaption></figure>

<figure><img src="/files/cGU9J5IzDOuWCIKIrFiw" alt=""><figcaption></figcaption></figure>

**Elastic File System Configuration**

* Add asset EFS mounts for both the UI and Job service

{% hint style="info" %}

* Both UI and Job task definitions must be updated before deploying new ECS services.
* Always deploy the latest task definition revision to apply changes.
  {% endhint %}

### Step 12: AskEdgi Feature Verification

* Once the application is up and running.
* Log in with the admin user.

Once the **OvalEdge application** is up and running, follow these steps.

1\. Navigate to System Settings\
**Administration** > **System Settings** > **AI Tab**

Update the following settings

1. askedgi.edition = Data Analytics
2. askedgiplus.enable = true
3. askedgi.ovaledge.recipe.connection.id = -1 (MySQL connector ID)

<figure><img src="/files/gSHFkS0U0UpeZTvUHrtV" alt=""><figcaption></figcaption></figure>

<figure><img src="/files/oqaqDBdz5SLWTinvMJlF" alt=""><figcaption></figcaption></figure>

4. Ai.config = (Developer will provide the KEY), and the remaining settings are the same as below.

<figure><img src="/files/ruZfEtMlQs9q8TeNFsll" alt=""><figcaption></figcaption></figure>

5. Ensure that all Marketplace details are correctly populated in the settings.

<figure><img src="/files/T4sXbRqE243Vs64LMReN" alt=""><figcaption></figcaption></figure>

Once done, the changes are logged out and relogged into the application.

**Navigate to the AskEdgi feature.**

<figure><img src="/files/9PxJtnGgwD9D9WitllPw" alt=""><figcaption></figcaption></figure>

1. It will take a few seconds to create a workspace.

<figure><img src="/files/foRNxBdjuJuMam7VnGGh" alt=""><figcaption></figcaption></figure>

<figure><img src="/files/Xzy2Zt7XCJVfPjINUyEQ" alt=""><figcaption></figcaption></figure>

Try to add the Catalog and Files Upload to the workspace.

<figure><img src="/files/vU7990d8EdSdC5yJkQWv" alt=""><figcaption></figcaption></figure>

<figure><img src="/files/lJUIdBdpDtsciQEpgMKo" alt=""><figcaption></figcaption></figure>

<figure><img src="/files/5YMbzz0s7QBYHImcoMvy" alt=""><figcaption></figcaption></figure>

Now, ask anything in the search option, and it should give an answer.

Once it gives the result, we are good with the complete setup.

If Workspace is not created:

* Check Insight Service logs.
* Validate IAM Role permissions.
* Verify user entry in the database:

```
select * from edgi_data_service_stack edss ;
delete * from edgi_data_service_stack edss;
delete from edgi_data_service_uptime_history ;
```

{% hint style="warning" %}
Do not run delete queries unless confirmed by the development team.
{% endhint %}

**Remove User Entry from Service Discovery**

* Navigate to **AWS Cloud Map** > **Service Discovery Resource**.
* Locate and delete the user entry associated with the application.
* Log back into the application to verify that the changes have taken effect.
* Check if the files have been uploaded or not.

<figure><img src="/files/IsAhBPsXCFdKYJPUOoKs" alt=""><figcaption></figcaption></figure>

{% hint style="info" %}

* Ensure that the Bridge Client certificate ZIP file is stored securely.
* Do not share AWS credentials externally; they are sensitive and allow direct access to S3.
* Always confirm that the S3 bucket exists and the region is correct before starting the Bridge Client.
  {% endhint %}

***

Copyright © 2025, OvalEdge LLC, Peachtree Corners, GA, USA.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.ovaledge.com/release8.1/askedgi/deployment-and-installation/ecs-variant/ecs-variant-2-new-customers.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.