Elasticsearch
It provides support for crawling database objects and profiling of sample data.
The connectivity to the Elasticsearch connector is via the Elasticsearch Rest Client. The connector currently supports the following versions of Elasticsearch:
The drivers used by the connector are given below:
Elasticsearch Rest Client
7.16.2and above
https://mvnrepository.com/artifact/org.elasticsearch.client/elasticsearch-rest-client/7.16.2
Note: Latest version 8.3.1
Pre-requisites
To use the Elasticsearch Connector, the details specified in the following section should be available.
An admin/service account for crawling and profiling.
The minimum privileges required are:
Connection validate
READ
Crawl schemas
READ
Crawl tables
READ
Profile schemas, tables
READ
Technical Specifications
The connector capabilities are shown below:
Crawling
Crawling
Tables
Table columns
All data types
Views
Profiling
Table Profiling
Row count, Columns count, View sample data
Supports all data types
View Profiling
Row count, Columns count, View sample data
View is treated as a table for profiling purposes
Column Profiling
Min, Max, Null count, distinct, top 50 values
Sample Profiling
Supported
Connection Details
The following connection setting details should be added for connecting to an Elasticsearch database:
Database Type
Elastic Search
License Type
Standard
Connection Name
Select a Connection name for the Elasticsearch database. The name that you specify is a reference name to easily identify the Elasticsearch database connection in OvalEdge.
Example: Elastic Search Connection DB1
Host/ IP /AES Endpoint
Database instance URL Example: https://search-esmckinsey-iuko6rg7qqyrc7ovf6au5bskxe.us-east-1.es.amazonaws.com/
Username
User account login credential
Password
Password for the user
Role-based Authentication
The following are the field attributes required for the connection of Elasticsearch for role-based authentication.
Region
Region of database
Is Role based Auth(Y/N)
provide role ARN
Role ARN
Region of database
Access key
Access Key
Secret key
Secret Key
Once connectivity is established, additional configurations for crawling and profiling can be specified:
Tables, views and Columns
If the checkbox is selected it will crawl the tables and columns in Elasticsearch.
Include Table
Regex
It catalogs the table based on the regex search pattern added.
Exclude Table Regex
It doesn’t catalog the table based on the regex search pattern added.
Profiler Settings:
Profile Type
Sample - Sample profiling will be performed based on sample profile size Disabled - profiling is disabled completely.
No. of threads
No of threads used for profiling
Query TimeOut
It is wait time for query response
Rowcount constraint
If it is checked the profiling performed based the rowcount limit
Profile rules
It will include/ exclude the tables to perform profiling based on regex search pattern added.
Copyright © 2025, OvalEdge LLC, Peachtree Corners GA USA
Was this helpful?