Elasticsearch

It provides support for crawling database objects and profiling of sample data.

The connectivity to the Elasticsearch connector is via the Elasticsearch Rest Client. The connector currently supports the following versions of Elasticsearch:

The drivers used by the connector are given below:

Driver / API	Version	Details
Elasticsearch Rest Client	7.16.2and above	https://mvnrepository.com/artifact/org.elasticsearch.client/elasticsearch-rest-client/7.16.2 Note: Latest version 8.3.1

Pre-requisites

To use the Elasticsearch Connector, the details specified in the following section should be available.

• An admin/service account for crawling and profiling.

• The minimum privileges required are:

Operation	Access Permission
Connection validate	READ
Crawl schemas	READ
Crawl tables	READ
Profile schemas, tables	READ

Technical Specifications

The connector capabilities are shown below:

Crawling

Feature	Supported Objects	Remarks
Crawling	Tables
Table columns	All data types
Views

Profiling

Feature	Support	Remarks
Table Profiling	Row count, Columns count, View sample data	Supports all data types
View Profiling	Row count, Columns count, View sample data	View is treated as a table for profiling purposes
Column Profiling	Min, Max, Null count, distinct, top 50 values
Sample Profiling	Supported

Connection Details

The following connection setting details should be added for connecting to an Elasticsearch database:

Property	Details
Database Type	Elastic Search
License Type	Standard
Connection Name	Select a Connection name for the Elasticsearch database. The name that you specify is a reference name to easily identify the Elasticsearch database connection in OvalEdge. Example: Elastic Search Connection DB1
Host/ IP /AES Endpoint	Database instance URL Example: https://search-esmckinsey-iuko6rg7qqyrc7ovf6au5bskxe.us-east-1.es.amazonaws.com/
Username	User account login credential
Password	Password for the user

Role-based Authentication

The following are the field attributes required for the connection of Elasticsearch for role-based authentication.

Property	Details
Region	Region of database
Is Role based Auth(Y/N)	provide role ARN
Role ARN	Region of database
Access key	Access Key
Secret key	Secret Key

Once connectivity is established, additional configurations for crawling and profiling can be specified:

Crawler configurations Settings	Details
Tables, views and Columns	If the checkbox is selected it will crawl the tables and columns in Elasticsearch.
Include Table Regex	It catalogs the table based on the regex search pattern added.
Exclude Table Regex	It doesn’t catalog the table based on the regex search pattern added.

Profiler Settings:

Profiler Settings	Details
Profile Type	Sample - Sample profiling will be performed based on sample profile size Disabled - profiling is disabled completely.
No. of threads	No of threads used for profiling
Query TimeOut	It is wait time for query response
Rowcount constraint	If it is checked the profiling performed based the rowcount limit
Profile rules	It will include/ exclude the tables to perform profiling based on regex search pattern added.

---

Copyright © 2025, OvalEdge LLC, Peachtree Corners GA USA