Data Quality Remediation Center

The Data Quality Remediation Center provides business users with a centralized workspace to review, track, and manage failed values generated by Data Quality Rules. It presents contextual information about each failure, including connector, schema, object, and attribute details, and identifies the user responsible for remediation.

The Remediation Center now contains two views:

• Summary – Displays an aggregated, high-level view of failed values grouped by object and attribute, along with remediation status counts such as New, In Progress, and Resolved.

• All – Displays detailed, record-level failed values and supports full remediation activities.

Together, these views give users both a consolidated overview of remediation workload and the ability to drill into individual failed records for corrective action.

Summary

The Summary view presents aggregated information grouped by object, attribute, rule, connector, and schema. This view allows users to quickly identify where failures have occurred, how many values require attention, and how remediation is progressing across the system. Users can then drill into the detailed records in the All tab when needed.

Status Bar

The status bar at the top of the Summary tab displays a high-level breakdown of remediation activity. It includes:

• New: The total number of failed values that have not been reviewed or updated. These records represent the initial workload that needs attention.

• In Progress: The number of failed values that are currently being investigated or remediated. These records indicate active work by the remediation assignee.

The status bar displays counts and percentages for both values, giving users a quick understanding of overall remediation progress.

Summary Table: The Summary table displays aggregated remediation information for each object and attribute involved in failed values. Each row represents a combination of contextual data and remediation counts.

Below is a description of each field displayed in the table:

• Object Type: Indicates the type of the data object, such as Table, File, Table Column, File Column, or Code.

• Connection Name: Displays the connector from which the data originates (for example, Snowflake, SQL Server, S3).

• Schema: Shows the schema associated with the failed values.

• Object: Displays the specific object where the rule was executed, such as a table or file.

• Attribute: Indicates the field or column within the object where the failure occurred.

• Scheme Rules Count: Shows the number of scheme-based rules applied to the object.

• Standalone Rules Count: Shows the number of standalone rules applied to the object.

• Failed Values Reported: The total number of failed values identified for the object or attribute.

• New: The number of failures currently in New status.

• In Progress: The number of failures currently marked as In Progress.

• Resolved: The count of failures that have been addressed and marked as Corrected, Allowed, or Void.

• First Reported On: Displays the timestamp when failed values were first identified for the object or attribute. This helps users understand when the remediation issue initially occurred.

• Last Reported On: Displays the timestamp when failed values were most recently detected for the object or attribute. This helps users track the latest occurrence of the failure and monitor ongoing issues.

Sorting, Searching, and Filtering: The Summary table supports sorting, searching, and filtering for fields where these options are enabled. These controls help users quickly locate specific records, refine the displayed list, and analyze remediation information efficiently.

All

The All View displays the complete list of failed values reported by Data Quality Rules. It provides a detailed, record-level view of each failure, including contextual information such as connector, schema, object, attribute, violation message, and remediation status. This view supports full remediation activities, allowing users to review individual failed rows, update their status, and take corrective actions as required.

• Failure ID: This column lists the ID of each failed value being reported to the Data Quality Remediation Center. This ID will be unique and help differentiate each value in the Data Quality Remediation Center. With the help of a search utility, users can search for a particular failed record. Additionally, the sorting option facilitates the arrangement of these values.

• Rule Name: This column plays a crucial role in informing users about the specific Data Quality Rule associated with a failed data object. This information is essential for identifying the rule to which a particular data object belongs, enabling users to pinpoint the source of the failure. The "DQ Rule Name" field is a hyperlink that allows users to navigate to the corresponding Data Quality Rule page. This redirection provides users with a deeper understanding of the failed value, offering insights into the details of the rule and its parameters, as well as guidance on effective remediation strategies.

• Violation Message: The purpose of a violation message is to provide information about why a particular data object has failed to meet the specified criteria. It serves several essential purposes for the technical and business users in identifying the root cause of the issue. The violation message is key for bridging the gap between users, fostering better communication, and enabling effective data quality management across various scenarios.

Configuration

The Violation message is configured at the Data Quality Rule level for functions supported by the Data Quality Remediation Center. There are two levels of configuration:

• Rule Level: The configured message at this level will be reflected for all the associated data objects within the rule.

• Object Level: Since multiple objects within a Data Quality Rule exist, different violation messages may apply to other objects. Therefore, with the object-level configuration, users can enter a violation message specific to that particular object.

The object-level configuration, if provided, will be reflected in the Data Quality Remediation Center, superseding the rule-level configuration.

The complete Violation Message in the Data Quality Remediation Center is displayed when a user hovers over the field and can be read from the tooltip.

The "Violation Message" and "Remediation Help" fields work hand in hand to create a comprehensive system for managing data quality. The violation message identifies and explains issues, while the remediation help field equips users with the tools and knowledge needed to address and resolve those issues effectively. They contribute to a more informed, collaborative, and efficient data quality management process.

Failed Values

• This column is of great significance under the Data Quality Remediation Center as it lists the failed values that are being reported. Each failed row is reported as a separate entry in the Data Quality Remediation Center.

• Each failed row value has an eye icon next to it, and clicking on this icon will open a pop-up listing the Primary and Secondary columns associated with the reported failed row. This will help users understand contextual information about the failed row and facilitate the remediation process.

Failed Values Counter

• The "Failed Values Counter" column in the Data Quality Remediation Center keeps track of how many values or rows have failed from a specific table column. For instance, if users see "8/41," it means it is the 8th failed value out of a total of 41 reported failed values. An important detail is the warning icon that appears next to the failed value when the actual count of failed values surpasses the reported count. This warning alerts users if there are more failed values than what's officially reported. To illustrate, let's consider a scenario where, at the Data Quality Rule level, the associated table columns have 100 rows, and the Max Failed Values Limit is set to 50. After a rule failure, all the rows failed, resulting in a total of 100 failed values. However, since the maximum configured value to be reported is 50, only the first 50 will be reported in the Data Quality Remediation Center. The warning icon alerts users to the potential issue. It signals that the actual count of failed values is higher than what's officially reported, prompting users to notice and address these additional failed records that may require attention.

Contextual Information

• Object Type: This column indicates the type of the data object, such as Table, File, Table Column, File Column, or Code.

• Connection Name: This column lists the connector name from which the failed records are being imported into the Data Quality Remediation Center.

• Schema: This column will list the schema name of the failed values.

• Object: This column displays the object name (Table, File, Code) under which the failed rows are present and are being reported to the Data Quality Remediation Center.

• Attribute: This column displays the Attribute Column name (Table Column, File Column) from which the failed rows are being reported to the Data Quality Remediation Center.

• Remediation Assignee: The Data Quality Remediation Center records failed values and supports the remediation process. The Remediation Assignee field displays the user responsible for addressing the failed records. By default, the Data Asset Custodian of the affected object is assigned as the Remediation Assignee, and this value can be updated when required.

• To update the Remediation Assignee:

  • Select the checkboxes for the failed records.

  • Click the nine dots menu.

  • Select Remediation Assignee Change.

  • The Update Remediation Assignee pop-up opens, allowing the user to search for and assign a User or Team to the selected failed records.

    
• Status: This field lists the status of the failed values with respect to the remediation process. The statuses below are displayed in the Data Quality Remediation Center:

  • New - This status indicates that the remediation process has not yet commenced. It signifies that no action has been taken to address the issue.

  • In Progress - This status reflects that the remediation process is actively underway. It is used when actions are being initiated to resolve the issue.

  • Corrected - This status is applied when the remediation has been successfully completed. It signifies that the issue has been resolved and is no longer a concern.

  • Void - This is used to classify a failed value as invalid. It is used when a rule mistakenly identifies values as failed and sends them to the Data Quality Remediation Center, even though they are not actually errors. In these cases, the values are marked as "Void" in the Data Quality Remediation Center, indicating that they should be ignored as errors.

  • Allowed - This is used when a value is a valid exception and requires review. When users designate selected value(s) as "Allowed," they are removed from the Data Quality Remediation Center because they are no longer necessary for remediation.

  • Note: When a user designates selected value(s) as “Corrected,” “Void,” or "Allowed," they are archived from the Data Quality Remediation Center as they have been acknowledged.

• To update the status:

  • Select the checkboxes for the failed records.

  • Click the nine dots menu.

  • Select Change Status.

  • The Change Remediation Center Status pop-up appears, allowing the user to select the required status from the dropdown. Once selected, the chosen status is applied to all selected failed records.

    

Remediation Help

• The purpose of this field is to provide users with information that can facilitate the remediation process.

• This field will have a remediation help icon, which, upon clicking, will display a pop-up showing the Corrective Action field and the Assistance SQL. The Corrective Action field provides users with actionable guidance and assistance on how to address and remediate data quality issues. This field is intended to offer practical steps, recommendations, or references that can help business users resolve the specific problems flagged by the data quality rules.

Configuration

The Corrective Action field is configured at the Data Quality Rule level, and the same content is being mapped to the Data Quality Remediation Center.

Below the Corrective Action field, users are provided with an Assistance SQL, which, when executed on the query sheet, lists all the failed values associated with the particular table column. The “Open To Query Sheet” option will help users open this query in the query sheet, offering them the flexibility to identify failed values.

• Monetary Value: This field is configured at the Data Quality Rule level and helps users understand the cost of each failed value reported as an independent entry in the Remediation Center.

• Criticality: The criticality field helps users understand the severity of the failed value and how critical it is for the user. With the monetary value and criticality fields, users can understand the impact a particular failed value will have.

• Execution Id: This column lists the rule execution id. This helps users in tracking the failed value being reported during a particular DQR execution.

• Object Execution Id: This column lists the object execution ID. This helps users in tracking the object-level execution details for the associated data object within that specific DQR execution.

• Assigned On: This displays the details, i.e., the timestamp when a particular failed value was reported to the remediation center.