How Data Quality Impacts Machine Learning Performance Across Domains
I investigated whether common data-cleaning techniques consistently improve machine-learning performance. Using logistic regression, I compared seven experimental conditions across income and healthcare datasets, finding that improvements involve trade-offs and depend on the dataset.
- Question
- How do common data-quality interventions affect classification performance?
- Domains
- UCI Adult income and Chronic Kidney Disease datasets
- Methods
- Logistic regression; seven conditions comparing missing-value treatment, duplicate removal, outlier clipping, category consistency, and class weighting
- Status
- Completed — report and notebooks available
Key findings
- 01
On the Adult income dataset, class weighting increased recall from 0.603 to 0.844, while precision fell from 0.733 to 0.554.
- 02
On the kidney dataset, outlier treatment corrected one classification error in the main test split. The improvement did not recur in the other tested splits.
- 03
Combining interventions did not consistently outperform simpler approaches. More cleaning did not automatically produce better predictions.
