Skip to main content
SpringerLink
Log in
Book cover

Discrete-Time Markov Chains

Two-Time-Scale Methods and Applications

  • Book
  • © 2005

Overview

Authors:
  1. G. George Yin

    1. Department of Mathematics, Wayne State University, Detroit, USA

    View author publications

    You can also search for this author in PubMed   Google Scholar

  2. Qing Zhang

    1. Department of Mathematics, University of Georgia, Athens, USA

    View author publications

    You can also search for this author in PubMed   Google Scholar

Part of the book series: Stochastic Modelling and Applied Probability (SMAP, volume 55)

This is a preview of subscription content, log in via an institution to check access.

Access this book

Log in via an institution
eBook USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book USD 109.00
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Other ways to access

Licence this eBook for your library

Institutional subscriptions

Table of contents (13 chapters)

  1. Front Matter

    Pages i-xix
    Download chapter PDF

  2. Prologue and Preliminaries

    1. Introduction, Overview, and Examples

      Pages 3-23

    2. Mathematical Preliminaries

      Pages 25-39

  3. Asymptotic Properties

    1. Asymptotic Expansions

      Pages 43-68

    2. Occupation Measures

      Pages 69-102

    3. Exponential Bounds

      Pages 103-118

    4. Interim Summary and Extensions

      Pages 119-129

  4. Applications

    1. Stability of Dynamic Systems

      Pages 133-158

    2. Filtering

      Pages 159-193

    3. Markov Decision Processes

      Pages 195-223

    4. LQ Controls

      Pages 225-250

    5. Mean-Variance Controls

      Pages 251-270

    6. Production Planning

      Pages 271-284

    7. Stochastic Approximation

      Pages 285-313

  5. Back Matter

    Pages 315-347
    Download chapter PDF
Back to top

Keywords

About this book

This book focuses on two-time-scale Markov chains in discrete time. Our motivation stems from existing and emerging applications in optimization and control of complex systems in manufacturing, wireless communication, and ?nancial engineering. Much of our e?ort in this book is devoted to designing system models arising from various applications, analyzing them via analytic and probabilistic techniques, and developing feasible compu- tionalschemes. Ourmainconcernistoreducetheinherentsystemcompl- ity. Although each of the applications has its own distinct characteristics, all of them are closely related through the modeling of uncertainty due to jump or switching random processes. Oneofthesalientfeaturesofthisbookistheuseofmulti-timescalesin Markovprocessesandtheirapplications. Intuitively,notallpartsorcom- nents of a large-scale system evolve at the same rate. Some of them change rapidly and others vary slowly. The di?erent rates of variations allow us to reduce complexity via decomposition and aggregation. It would be ideal if we could divide a large system into its smallest irreducible subsystems completely separable from one another and treat each subsystem indep- dently. However, this is often infeasible in reality due to various physical constraints and other considerations. Thus, we have to deal with situations in which the systems are only nearly decomposable in the sense that there are weak links among the irreducible subsystems, which dictate the oc- sional regime changes of the system. An e?ective way to treat such near decomposability is time-scale separation. That is, we set up the systems as if there were two time scales, fast vs. slow. xii Preface Followingthetime-scaleseparation,weusesingularperturbationmeth- ology to treat the underlying systems.

Reviews

From the reviews:

"Discrete-time Markov chains are the basic building blocks for understanding random dynamic phenomena, in preparation for more complex situations. … the book is a research monograph based largely on the author’s own work. … The book does … fill an important niche in the literature on singularly perturbed Markov chains. … the book will be useful to applied probabilities and engineers who deal with such systems. Other than this, the book’s primary audience is other researchers in singulary perturbed Markov chains." (IEEE Control Systems Magazine, December, 2005)

Authors and Affiliations

  • Department of Mathematics, Wayne State University, Detroit, USA

    G. George Yin

  • Department of Mathematics, University of Georgia, Athens, USA

    Qing Zhang

Bibliographic Information

Publish with us

Policies and ethics

Back to top

Search

Navigation