Comparative Analysis of Data Preprocessing Techniques to Produce Balanced and Quality HR Performance Appraisal Data Using Databricks Medallion Architecture

Abstract

Data preprocessing plays a significant role in improving the quality, reliability, and predictive capability of machine learning models. In Human Resource (HR) analytics, employee performance appraisal datasets often suffer from issues such as missing values, imbalance, noise, outliers, and inconsistent scaling. These challenges reduce model accuracy and affect decision-making quality. This research paper presents a comparative analysis of various preprocessing techniques applied to employee performance appraisal datasets obtained from Kaggle. The study evaluates preprocessing methods including missing value treatment, normalization, standardization, outlier handling, feature encoding, and balancing approaches such as Random Oversampling, Random Undersampling, and SMOTE. The paper further proposes a conceptual preprocessing framework based on the Databricks Medallion Architecture (Bronze-Silver-Gold layers) for scalable HR analytics. Experimental findings indicate that balanced preprocessing using SMOTE combined with normalization and feature engineering significantly improves classification performance, data consistency, and analytical reliability. The study demonstrates the importance of preprocessing in producing high-quality balanced datasets for talent prediction and employee performance analytics.