In as much detail as possible, describe the problem or experience related to your idea. Please provide the context of what you were trying to do and include specific examples or workarounds:
When transforming data with ADO, it's difficult to validate the integrity of the data:
- Column profiling tools would be useful. It would be good to be able to toggle the profile on the first 1000 rows or entire dataset.
- Uniqueness analysis. Count of distinct and duplicate values in the column. Flag when a column marked as a key contains duplicates.
- Data quality summary. Percentage of valid, empty, and error values in the column. Identify null or missing data before loading.
- Value distribution. Frequency chart showing the top values and their count. Spot outliers, gaps in sequences, or unexpected concentrations.
- Basic statistics. For numeric columns: min, max, average. For text: character length range. Quickly validate assumptions about the data.
- Data type consistency. Identify values that don't match the expected data type (text in a numeric column, for example).
- Duplicate record flagging identify columns with duplicate values
- The ability to sort by a column, whilst building the transformation view would be useful
- Currently you can sort when previewing a transformation view, but not whilst building one
- You have to go out of the edit, preview and sort if you want to be able to do this
Who is this impacting? (ex. model builders, solution architects, partners, admins, integration experts, business/end users, executive-level business users)
What would your ideal solution be? How would it add value to your current experience?
- Column profiling tools
- Sort ability
Please include any images to help illustrate your experience.
