Overview
- Covers how to transition from Python-based ML models to PySpark-based large scale models
- Covers how to automate your data workflow using Airflow
- Explains the end-to end machine learning pipeline for model prediction
Access this book
Tax calculation will be finalised at checkout
Other ways to access
Table of contents (9 chapters)
Keywords
About this book
Master the new features in PySpark 3.1 to develop data-driven, intelligent applications. This updated edition covers topics ranging from building scalable machine learning models, to natural language processing, to recommender systems.
Machine Learning with PySpark, Second Edition begins with the fundamentals of Apache Spark, including the latest updates to the framework. Next, you will learn the full spectrum of traditional machine learning algorithm implementations, along with natural language processing and recommender systems. You’ll gain familiarity with the critical process of selecting machine learning algorithms, data ingestion, and data processing to solve business problems. You’ll see a demonstration of how to build supervised machine learning models such as linear regression, logistic regression, decision trees, and random forests. You’ll also learn how to automate the steps using Spark pipelines, followed by unsupervised models such as K-means and hierarchical clustering. A section on Natural Language Processing (NLP) covers text processing, text mining, and embeddings for classification. This new edition also introduces Koalas in Spark and how to automate data workflow using Airflow and PySpark’s latest ML library.
After completing this book, you will understand how to use PySpark’s machine learning library to build and train various machine learning models, along with related components such as data ingestion, processing and visualization to develop data-driven intelligent applications
What you will learn:
- Build a spectrum of supervised and unsupervised machine learning algorithms
- Use PySpark's machine learning library to implement machine learning and recommender systems
- Leverage the new features in PySpark’s machine learning library
- Understand data processing using Koalas in Spark
- Handle issues around feature engineering, class balance, bias andvariance, and cross validation to build optimally fit models
Who This Book Is For
Data science and machine learning professionals.Authors and Affiliations
About the author
Bibliographic Information
Book Title: Machine Learning with PySpark
Book Subtitle: With Natural Language Processing and Recommender Systems
Authors: Pramod Singh
DOI: https://doi.org/10.1007/978-1-4842-7777-5
Publisher: Apress Berkeley, CA
eBook Packages: Professional and Applied Computing, Apress Access Books, Professional and Applied Computing (R0)
Copyright Information: Pramod Singh 2022
Softcover ISBN: 978-1-4842-7776-8Published: 09 December 2021
eBook ISBN: 978-1-4842-7777-5Published: 08 December 2021
Edition Number: 2
Number of Pages: XVIII, 220
Number of Illustrations: 202 b/w illustrations
Topics: Machine Learning, Python, Artificial Intelligence, Open Source