Contents
Preface
Chapter l Basis of probability 1
1.1 Random ovcnts 1
1.1.1 Random experiments 2
1.1.3 Calculation among random evcnts 6
1.1.4 u-algebras 10
1.2.1 What is probability 12
1.2.2 Properties of probability 14
1.2.3 Two fundamental probability models 16
1.2.4 Conditional probability and three important formulas 19
1.2.5 Independence of events 23
1.3 Random variables 25
1.3.1 Definition 25
1.3.2 Cumulative distribution functions 27
1.3.4 Multiple-dimensional random variablcs 34
1.3.5 Distributions of functions of random variables 38
1.3.6 Conditional distributions and indepenclences 44
1.3.70rder statistics 47
1.4 Numerical characteristics 50
1.4.1 Definition of mathematical expectations 50
1.4.2Properties 56
1.4.3 0ther numerical characteristics of random variables 57
1.4.4Conditional mathematical expectations 73
1.5 Limiting thcoroms 76
1.5.1 Convcrgencc of random variablcs 76
1.5.2Law of large numbers 80
1.5.3 Central limit thcorcms 84
Answcrs or tips for Excrciscl 88
Chapter 2 Stochastic processes 95
2.1 Definition and classification 95
2.1.1 Definition 96
2.1.3 Examples 99
2.2 Statistical laws of stochastic processes 102
2.2.1 Finite dimensional distribution functions 102
2.2.2Kolmogorov's theorem 103
2.3 Mcasurcmcnts of stochastic proccsscs 104
2.3.1 Measurements for one stochastic process 104
2.3.2 Measurements for two stochastic processes 107
2.4 Further comments 108
Excrcisc 2 109
Answors or tips for Excrcisc 2 111
Chapter 3 Poisson processes 112
3.1 Definition and measurements 112
3.1.1 Definition 112
3.1.2 Measurements 123
3.2 Waiting timcs and intcrarrival times 126
3.2.1 Waiting times 127
3.2.2 Interarrival times 130
3.3 Conditional distributions 131
3.4 Extcnsions of Poisson procosscs 136
3.4.1 Non-homogencous Poisson proccsscs 137
3.4.2 Compound Poisson processes 142
3.4.3 Conditional Poisson processes 145
3.4.4 Renewal processes 147
Excrcis0 3 152
Answers or tips for Exercise 3 154
Chapter4Discrete-time Markovchains 157
4.1 Definition of discrete-time Markov chains 157
4.1.1 Definition 159
4.1.2 Chapman-Kolmogorov equations 163
4.2 Finite-dhnensional distributions 165
4.3 Propcrtios of a singlo statc 168
4.3.2 Transience and recurrence 169
4.4 Decomposition of state space 177
4.4.1 Equivalencc rclation 177
4.4.2Decomposition of statc space 181
4.5 Asymptotic behaviors of transition probabilities p(.j ) 184
4.5.1 Case one: state j is transient or null-recurrent 185
4.5.2 Case two: state j is positive-recurrent 186
4.6 Stationary distributions 196
4.6.1 Definition of stationary distributions 196
4.6.2 How many stationary distributions a Markov chain may have? 199
4.6.3 Rates of convergence to stationary distributions 201
4.6.4 Stationary distributions of a ccnsored Markov chain 201
4.6.5 Quasi-stationary distributions 203
4.7 Rovcrsiblc Markov chains 206
Exercise 4 209
Answers or tips for Exercise 4 211
Chapter5Continuous-timeMarkovchains 213
5.1 Dofinitionof continuous-timc Markov chains 213
5.1.1 Definition 213
5.1.2 Chapman-Kolmogorov equations 218
5.2 Finite-dhnensional distributions 218
5.3 Q-matriccs 220
5.4 Kolmogorov difforcntial cquations 227
5.5 Asymptotic behaviors 236
5.5.1 Transience and recurrence 237
5.5.2 Limiting results 239
5.5.3 Stationary distributions 240
5.6 Birth and death processes 243
Exercise 5 249
Answers or tips for Exercise 5 251
Chapter 6Simple Markovian queueing models 256
6.1 Torminology and notation 256
6.2 Little's law and PASTA property 259
6.2.1 Littlc's law 259
6.2.2PASTA property 260
6.3M/M/l queueing model 261
6.3.1 Stationary distribution of queue length 261
6.3.2 Distributions of sojourn times and waiting times 266
6.3.3 Busy period distribution 270
6.3.4 Departure process 273
6.4 M/M/n, and state dependent M/M/l queueing model 275
6.4.1 M/M/n, queucing modcl 275
6.4.2 State dcpendentM/M/lqueucing modcl 276
6.5 M xlM/l queueing model 279
6.6 M/G/l queueing model 281
6.6.1 Embedded Markov chain 281
6.6.2 M/E7-/1 qucueing model 286
Exercise 6 290
Answers or tips for Exercise 6 292
Chapter 7Stationary processes 295
7.1.1 Strict-sense stationary processes 295
7.1.2Wide-sense stationary processes 298
7.2 Analytic propcrtics of widc-sonso stationary proccsscs 302
7.3 Corrclation functions and thcir spcctra 312
7.3.1 Properties of correlation functions 312
7.3.2 Spectral density functions 314
7.3.3 Propcrtics of spectral density functions 323
7.3.4 Continuous-time whitc noisc processes 325
7.5 Passing through a linear time-invariant system 335
Excrcisc 7 345
Answcrs or tips for Excrcisc 7 346