Skip to main content
  • Book
  • © 2020

SQL Server Data Automation Through Frameworks

Building Metadata-Driven Frameworks with T-SQL, SSIS, and Azure Data Factory

Apress
  • Shows how to design and implement automation on-premises and in Azure Data Factory cloud solutions
  • Provides patterns and examples for creating enterprise automation using free tools from Microsoft
  • Demonstrates and promotes techniques that are critical to enterprise DevOps

Buy it now

Buying options

eBook USD 34.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book USD 44.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Other ways to access

This is a preview of subscription content, log in via an institution to check for access.

Table of contents (11 chapters)

  1. Front Matter

    Pages i-xx
  2. Stored Procedure-Based Database Frameworks

    1. Front Matter

      Pages 1-1
    2. Stored Procedures 101

      • Andy Leonard, Kent Bradshaw
      Pages 3-18
    3. Automation with Stored Procedures

      • Andy Leonard, Kent Bradshaw
      Pages 19-26
    4. Stored Procedure Orchestrators

      • Andy Leonard, Kent Bradshaw
      Pages 27-32
    5. A Stored Procedure–Based Metadata-Driven Framework

      • Andy Leonard, Kent Bradshaw
      Pages 33-56
  3. SSIS Frameworks

    1. Front Matter

      Pages 57-57
    2. A Simple, Custom, File-Based SSIS Framework

      • Andy Leonard, Kent Bradshaw
      Pages 59-96
    3. Framework Execution Engine

      • Andy Leonard, Kent Bradshaw
      Pages 97-158
    4. Framework Logging

      • Andy Leonard, Kent Bradshaw
      Pages 159-206
    5. Azure-SSIS Integration Runtime

      • Andy Leonard, Kent Bradshaw
      Pages 207-240
    6. Deploy a Simple, Custom, File-Based Azure-SSIS Framework

      • Andy Leonard, Kent Bradshaw
      Pages 241-299
    7. Framework Logging in ADF

      • Andy Leonard, Kent Bradshaw
      Pages 301-353
    8. Fault Tolerance in the ADF Framework

      • Andy Leonard, Kent Bradshaw
      Pages 355-383
  4. Back Matter

    Pages 385-391

About this book

Learn to automate SQL Server operations using frameworks built from metadata-driven stored procedures and SQL Server Integration Services (SSIS). Bring all the power of Transact-SQL (T-SQL) and Microsoft .NET to bear on your repetitive data, data integration, and ETL processes. Do this for no added cost over what you’ve already spent on licensing SQL Server. The tools and methods from this book may be applied to on-premises and Azure SQL Server instances. The SSIS framework from this book works in Azure Data Factory (ADF) and provides DevOps personnel the ability to execute child packages outside a project—functionality not natively available in SSIS.


Frameworks not only reduce the time required to deliver enterprise functionality, but can also accelerate troubleshooting and problem resolution. You'll learn in this book how frameworks also improve code quality by using metadata to drive processes. Much of the work performed by data professionals can be classified as “drudge work”—tasks that are repetitive and template-based. The frameworks-based approach shown in this book helps you to avoid that drudgery by turning repetitive tasks into "one and done" operations. Frameworks as described in this book also support enterprise DevOps with built-in logging functionality.


What You Will Learn
  • Create a stored procedure framework to automate SQL process execution
  • Base your framework on a working system of stored procedures and execution logging
  • Create an SSIS framework to reduce the complexity of executing multiple SSIS packages
  • Deploy stored procedure and SSIS frameworks to Azure Data Factory environments in the cloud



Who This Book Is For


Database administrators and developers who are involved in enterprise data projects built around stored procedures and SQL Server Integration Services (SSIS). Readersshould have a background in programming along with a desire to optimize their data efforts by implementing repeatable processes that support enterprise DevOps.



Authors and Affiliations

  • Farmville, USA

    Andy Leonard

  • Providence Forge, USA

    Kent Bradshaw

About the authors

Andy Leonard is Chief Data Engineer at Enterprise Data & Analytics, creator and Data Philosopher at DILM (Data Integration Lifecycle Management) Suite, an Azure Data Factory and SQL Server Integration Services trainer and consultant, and a BimlHero. He is a SQL Server database and data warehouse developer, community mentor, engineer, and farmer. Andy is co-author of SQL Server Integration Services Design Patterns, Data Integration Life Cycle Management with SSIS, and The Biml Book.


Kent Bradshaw is the founder or Tudor Data Solutions, LLC. With over 40 years of IT experience, he is a SQL Server database/ETL developer and database architect with a background is in Medicaid claims, public schools, government, retail, and insurance systems. In 2011, Kent founded Tudor Data Solutions, LLC to pursue new development opportunities which led to his association with Andy Leonard and Enterprise Data & Analytics. In 2017, Kent received the MPP certification for Data Science.



Bibliographic Information

Buy it now

Buying options

eBook USD 34.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book USD 44.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Other ways to access