Data Quality Remediation Center
The Data Quality Remediation Center is intended for business users. It acts as an error-trapping table for business users to get a detailed view of the Failed Values coming from data objects associated with a Data Quality Rule.
The Data Quality Remediation Center lists the failed values coming from the data objects associated with executed data quality rules; provides contextual information (connector name, schema name, object name) about them, and allocates a user who works on carrying out the Remediation Process.
The Data Quality Remediation Center will list the following columns:
Failure ID: This column lists the ID of each failed value being reported into the Data Quality Remediation Center. This ID will be unique and help differentiate each value in the Data Quality Remediation Center. With the help of a search utility, users can search for a particular failed record. Also, the sorting option helps in arranging these values.
DQ Rule Name: This column plays a crucial role in informing users about the specific Data Quality Rule associated with a failed data object. This information is important for tracking the rule to which a particular data object belongs, allowing users to address the source of the failure. The "DQ Rule Name" field is a hyperlink, enabling users to navigate to the corresponding Data Quality Rule page. This redirection provides users with a deeper understanding of the failed value, offering insights into the details of the rule and its parameters and potentially offering guidance on effective remediation strategies.
Violation Message: The purpose of a violation message is to provide information about why a particular data object has failed to meet the specified criteria. It serves several essential purposes for the technical and business users in identifying the root cause of the issue. The violation message is key for bridging the gap between users, fostering better communication, and enabling effective data quality management across various scenarios.
Configuration
The Violation message is configured at the Data Quality Rule level for the functions that are supported for the Data Quality Remediation Center. There are two levels of configuration:
Rule Level: The configured message at this level will be reflected for all the associated data objects within the rule.
Object Level: As multiple objects within a Data Quality Rule exist, there can be different violation messages for different objects. Therefore, with the object level configuration, users can enter the violation message specific to that particular object.
The object-level configuration, if provided, will be reflected in the Data Quality Remediation Center, and the rule-level configuration will be superseded.
The complete Violation Message in Data Quality Remediation Center will be viewed when a user hovers over the field and can be read from the tooltip.
The "Violation Message" and "Remediation Help" fields work hand in hand to create a comprehensive system for managing data quality. The violation message identifies and explains issues, while the remediation help field equips users with the tools and knowledge needed to address and resolve those issues effectively. They contribute to a more informed, collaborative, and efficient data quality management process.
Failed Values
This column is of great significance under the Data Quality Remediation Center as it lists the failed values that are being reported. Each failed row is reported as a separate entry in the Data Quality Remediation Center.
Each failed row value has an eye icon next to it, and clicking on this will open the popup listing the Primary and Secondary columns associated with the reported failed row, which will help the users understand contextual information about the failed row and help them in the remediation process.
Failed Values Counter
The "Failed Values Counter" column in the Data Quality Remediation Center keeps track of how many values or rows have failed from a specific table column. For instance, if users see "8/41," it means it is the 8th failed value out of a total of 41 reported failed values. An important detail is the warning icon that appears next to the failed value when the actual count of failed values surpasses the reported count. This warning alerts users if there are more failed values than what's officially reported. To illustrate, let us consider a scenario where, at the Data Quality Rule level, associated table columns have 100 rows and the Max Failed Values Limit is set to 50. After a rule failure, all the rows failed, resulting in a total of 100 failed values. However, since the maximum configured value to be reported is 50, only the first 50 will be reported in the Data Quality Remediation Center. The warning icon steps in to make users aware of the potential issue. It signals that the actual count of failed values is higher than what's officially reported, prompting users to notice and address these additional failed records that may require attention.
Contextual Information
Connector: This column lists the connector name from where the failed records are coming into the Data Quality Remediation Center.
Schema: This column will list the schema name of the failed values. Object: This column displays the Table Column name under which the failed rows are present and are being reported to the Data Quality Remediation Center.
Attribute: This column displays the Table Name from which the failed rows are being reported into the Data Quality Remediation Center.
Remediation Assignee: The purpose of this Data Quality Remediation Center is to record the failed values and help in carrying out the remediation. Hence, the Remediation Assignee field lists the designated person who needs to be notified as well as supposed to be working on fixing the failed records.By default, the Data Asset Custodian of the failed record is assigned as the Remediation Assignee, which can be updated. It is important to note that only Author license users can be updated as remediation assignees.
Current Status: This field lists the status of the failed values with respect to the remediation process. The below statuses are displayed in the Data Quality Remediation Center:
New - This status indicates that the remediation process has not yet commenced. It signifies that no action has been taken to address the issue.
In Progress - This status reflects that the remediation process is actively underway. It is used when actions are being initiated to resolve the issue.
Corrected - This status is applied when the remediation has been successfully completed. It signifies that the issue has been resolved and is no longer a concern.
Void - This is used to classify a failed value as invalid. It is used when a rule mistakenly identifies values as failed and sends them to the Data Quality Remediation Center, even though they are not actually errors. In these cases, the values are marked as "Void" in the Data Quality Remediation Center, indicating that they should be ignored as errors.
Allowed - This is used when a value is a valid exception and requires review. When users designate selected value(s) as "Allowed," they are removed from the Data Quality Remediation Center because they are no longer necessary for remediation.
Note: When a user designates selected value(s) as “Corrected,” “Void,” or "Allowed," they are archived from the Data Quality Remediation Center as they have been acknowledged.
Remediation Help
The purpose of this field is to provide users with the information that can be used to facilitate the remediation process.
This field will have an remediation help icon, which, upon clicking, will display a pop-up showing the Corrective Action field and the Assistance SQL.The Corrective Action field provides actionable guidance and assistance to users on how to address and remediate data quality issues. This field is intended to offer practical steps, recommendations, or references that can help business users resolve the specific problems flagged by the data quality rules. Users can tag Data Stories within this field, clicking on which will redirect them to the respective data story, helping them in carrying out the remediation.
Configuration
The Corrective Action field is configured at the Data Quality Rule level, and the same content is being mapped to the Data Quality Remediation Center.
Below the Corrective Action field, users are provided with an Assistance SQL, which, when executed on the query sheet, will list all the failed values falling under the particular table column. The “Open To Query Sheet” option will help users open this query in the query sheet and offer users the flexibility to identify the failed values.
Monetary Value: This field is configured at the Data Quality Rule level and will help the users understand the cost of each failed value being reported as an independent entry in the Remediation Center.
Criticality: The criticality field helps users understand the severity of the failed value and how critical is a failed value for the users. With monetary value and criticality fields, users can understand the impact a particular failed value will cause.
Rule Execution Id: This column lists the rule execution id. This helps users in tracking the failed value being reported during a particular DQR execution.
Object Execution Id: This column lists the object execution id. This helps users in tracking the
Assigned On: This displays the details, i.e., the timestamp when a particular failed value was reported to the remediation center.
Was this helpful?