Data Quality Logs
Definition
This module is used to record and display execution logs and evaluation results for Data Quality evaluation tasks, helping users track each task execution, view specific quality rule results, and manage discovered issue data, ensuring quality issues can be queried, managed, and controlled.
Module Features
- 📄 Task execution logs: view the execution status, duration, scheduling method, executor, and other information for each quality task;
- 📊 Rule evaluation results: view execution results for each configured quality rule under a task, including abnormal count, total count, abnormal ratio, and more;
- 🧯 Issue data management:
- The list displays abnormal data found by each quality rule, including evaluation fields, error descriptions, and data samples;
- Supports marking individual or batch data as ignored;
- Supports adding remarks to issue data, manual handling, resetting status, and other operations;
- Supports multi-condition filtering and quick search.
Notes
We recommend regularly reviewing issue data handling records to avoid incorrect handling or omission risks. Log and report data should support one-to-one correspondence with quality tasks to ensure traceability.
Scope
For quality management personnel and data owners, this module is used for visualizing the evaluation process and continuously tracking and handling issue data. Together with quality evaluation tasks, it implements a closed-loop quality management process from issue discovery to handling.
1. Quality Task Log
Click Data Assets -> Data Quality -> Quality Task Log to open the Quality Task Log page. The page displays a quality task log list including ID, task name, quality score, execution status, issue data, start time, end time, and other fields.
2. Details
Select the quality task log whose details need to be viewed, click Details in the operation column, and you can view the details of the quality task log.
2.1 Governance Data Volume Change Trend
Select a statistical time range of the last 7 days, last 15 days, or last 30 days to view the data volume change trend.
2.2 Rule List
The details page includes a rule list. The list includes evaluation name, database name, field name/Chinese name, quality dimension, audit name, issue data volume ratio, and other related fields. Click View Issue Data in the operation column of the rule list to open the Issue Data Handling dialog.
2.2.1 Issue Data Handling
The Issue Data Handling dialog includes an issue data list. The list includes issue data, evaluation fields, severity, error description, recommended fix, evaluation date, and other related fields. Click Ignore, Add Remarks, or Handle in the operation column to complete the corresponding operation.
