Statistical Learning from a Regression Perspective

1. Front Matter

   Pages i-xxvi

   PDF

2. Statistical Learning as a Regression Problem

   • Richard A. Berk

   Pages 1-72

3. Splines, Smoothers, and Kernels

   • Richard A. Berk

   Pages 73-156

4. Classification and Regression Trees (CART)

   • Richard A. Berk

   Pages 157-211

5. Bagging

   • Richard A. Berk

   Pages 213-232

6. Random Forests

   • Richard A. Berk

   Pages 233-295

7. Boosting

   • Richard A. Berk

   Pages 297-337

8. Support Vector Machines

   • Richard A. Berk

   Pages 339-359

9. Neural Networks

   • Richard A. Berk

   Pages 361-399

10. Reinforcement Learning and Genetic Algorithms

    • Richard A. Berk

    Pages 401-413

11. Integrating Themes and a Bit of Craft Lore

    • Richard A. Berk

    Pages 415-425

12. Back Matter

    Pages 427-433

    PDF

This textbook considers statistical learning applications when interest centers on the conditional distribution of a response variable, given a set of predictors, and in the absence of a credible model that can be specified before the data analysis begins. Consistent with modern data analytics, it emphasizes that a proper statistical learning data analysis depends in an integrated fashion on sound data collection, intelligent data management, appropriate statistical procedures, and an accessible interpretation of results. The unifying theme is that supervised learning properly can be seen as a form of regression analysis. Key concepts and procedures are illustrated with a large number of real applications and their associated code in R, with an eye toward practical implications. The growing integration of computer science and statistics is well represented including the occasional, but salient, tensions that result. Throughout, there are links to the big picture.

The third edition considers significant advances in recent years, among which are:

• the development of overarching, conceptual frameworks for statistical learning;
• the impact of  “big data” on statistical learning;
• the nature and consequences of post-model selection statistical inference;
• deep learning in various forms;
• the special challenges to statistical inference posed by statistical learning;
• the fundamental connections between data collection and data analysis;
• interdisciplinary ethical and political issues surrounding the application of algorithmic methods in a wide variety of fields, each linked to concerns about transparency, fairness, and accuracy.

This edition features new sections on accuracy, transparency, and fairness, as well as a new chapter on deep learning. Precursors to deep learning get an expanded treatment. The connections between fitting and forecasting are considered in greater depth. Discussion of the estimation targets for algorithmic methods is revised and expanded throughout to reflect the latest research. Resampling procedures are emphasized. The material is written for upper undergraduate and graduate students in the social, psychological and life sciences and for researchers who want to apply statistical learning procedures to scientific and policy problems.

• classification
• random forests
• support vector machines
• machine learning
• boosting
• bagging
• decision trees
• support vector machines
• neural networks
• generalized additive model

“It could readily be a textbook for an applications-focused course at the graduate level as each chapter comes with exercises … . Examples with accompanying code also appear throughout the chapters which provide a scaffold for getting started … . Berk’s pragmatic advice will serve a wide audience from practitioners to educators to students.” (Sara Stoudt, MAA Reviews, December 12, 2021)

• Department of Criminology, Schools of Arts and Sciences, University of Pennsylvania, Philadelphia, USA

  Richard A. Berk

Richard Berk is Distinguished Professor of Statistics Emeritus at UCLA and currently a Professor at the University of Pennsylvania in the Department of Statistics and in the Department of Criminology. He is an elected fellow of the American Statistical Association and the American Association for the Advancement of Science and has served in a professional capacity with a number of organizations such as the Committee on Applied and Theoretical Statistics for the National Research Council and the Board of Directors of the Social Science Research Council. His research has ranged across a variety of statistical applications in the social and natural sciences.

Policies and ethics