Decision Tree Frameworks for Enhanced Teaching Effectiveness Evaluation with Transparent Data Driven Educational Decision-Making

Abstract

This study aimed to develop and validate an interpretable decision tree framework for automatically classifying unstructured student evaluation comments according to standardized teaching effectiveness dimensions used in Philippine State Universities and Colleges, while maintaining transparency and practical utility for institutional decision-making. A mixed-methods sequential exploratory design was employed. Qualitative thematic analysis of 250 student comments established a coding framework aligned with four teaching dimensions (Commitment, Knowledge of Subject, Teaching for Independent Learning, Management of Learning). Three expert evaluators achieved inter-rater reliability of κ=0.81 across 1,200 manually coded comments. Multiple decision tree algorithms (CART, C4.5, Random Forest) were trained on 840 comments, validated on 180, and tested on 180, with hyperparameter optimization via 5-fold cross-validation. The final dataset comprised 4,410 comments from 347 course sections across four colleges at a Philippine state university during second semester 2023-2024. Implementation testing assessed efficiency gains and user satisfaction. Random Forest achieved optimal performance with 84.2% overall accuracy, ranging from 77.6% (Commitment) to 88.1% (Knowledge of Subject) across dimensions. Expert validation showed substantial agreement (κ=0.78). Feature importance analysis identified "clear" (0.094), "helpful" (0.087), and "engaging" (0.081) as top predictors. Implementation testing demonstrated 74% reduction in analysis time while maintaining quality (4.0-4.3/5.0 ratings). High-confidence decision rules (84.7-93.2% confidence) provided transparent classification logic. Decision tree frameworks enable efficient, transparent analysis of qualitative teaching feedback aligned with institutional evaluation criteria, supporting evidence-based faculty development in resource-constrained Philippine higher education contexts