Artificial Intelligence Applications in Engineering (MUH-920), week 7 of 14: interactive lab

Regression studio: Complexity, validation and leakage

Prof. Dr. Utku Kose, Süleyman Demirel University

Part A generates strength data from a known law, shaped after Abrams' water-cement rule [1], adds noise, and lets polynomial, nearest-neighbour and tree regressors fit it; training error, error on new data and a five-fold validation curve show where overfitting starts [2]. Part B is a leakage detective with six candidate features from engineering projects [3].

Part A: Fit, validate, compare

Five-fold validation curve over the complexity setting of the chosen model. The dashed line marks the current setting.

Part B: Leakage detective

References

[1] Abrams, D. A. (1918). Design of Concrete Mixtures (Bulletin 1). Structural Materials Research Laboratory, Lewis Institute, Chicago.

[2] Hastie, T., Tibshirani, R., & Friedman, J. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction (2nd ed.). Springer. https://doi.org/10.1007/978-0-387-84858-7

[3] Kaufman, S., Rosset, S., Perlich, C., & Stitelman, O. (2012). Leakage in data mining: Formulation, detection, and avoidance. ACM Transactions on Knowledge Discovery from Data, 6(4), 15. https://doi.org/10.1145/2382577.2382579

[4] Hoerl, A. E., & Kennard, R. W. (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12(1), 55-67. https://doi.org/10.1080/00401706.1970.10488634

[5] Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5-32. https://doi.org/10.1023/A:1010933404324