SEMI-MARKOV MODELS
Theory and Applications
SEMI-MARKOV MODELS
Theory and Applications
Edited by
Jacques Janssen
Free University of Brussels
Brussels, Belgium
Springer Science+Business Media, LLC
Library of Congress Cataloging in Publication Data
International Symposium on Semi-Markov Processes and Their Applications (1984:
Brussels, Belgium)
Semi-Markov models.
"Proceedings of an International Symposium on Semi-Markov Processes and Their
Applications, held June 4-7, 1984, in Brussels, Belgium"—T.p. verso.
Bibliography: p.
1. Markov, processes—Congresses. 2. Renewal theory—Congresses. I. Janssen, Jac
. II. Title.
ques, 1939-
QA274.7.I58 1984
ISBN 978-1-4899-0576-5
519.2'33
86-12389
ISBN 978-1-4899-0576-5
DOI 10.1007/978-1-4899-0574-1
ISBN 978-1-4899-0574-1 (eBook)
Proceedings of an International Symposium on Semi-Markov Processes and
Their Applications, held June 4-7, 1984, in Brussels, Belgium
© 1986 Springer Science+Business Media New York
Originally published by Plenum Press, New York in 1986
Softcover reprint of the hardcover 1st edition 1986
All rights reserved
No part of this book may be reproduced, stored in a retrieval system, or transmitted
in any form or by any means, electronic, mechanical, photocopying, microfilming,
recording, or otherwise, without written permission from the Publisher
PREFACE
This book is the result of the International Symposium on Semi
Markov Processes and their Applications held on June 4-7, 1984 at
the Universite Libre de Bruxelles with the help of the FNRS (Fonds
National de la Recherche Scientifique, Belgium), the Ministere de
l'Education Nationale (Belgium) and the Bernoulli Society for Mathe
matical Statistics and Probability.
This international meeting was planned to make a state of the
art for the area of semi-Markov theory and its applications, to
bring together researchers in this field and to create a platform
for open and thorough discussion.
Main themes of the Symposium are the first ten sections of this
book. The last section presented here gives an exhaustive biblio
graphy on semi-Markov processes for the last ten years.
Papers selected for this book are all invited papers and in
addition some contributed papers retained after strong refereeing.
Markov additive processes and regenerative systems
Semi-Markov decision processes
Sections are
I.
II.
III. Algorithmic and computer-oriented approach
Semi-Markov models in economy and insurance
IV.
V.
Semi-Markov processes and reliability theory
VI.
Simulation and statistics for semi-Markov processes
VII. Semi-Markov processes and queueing theory
VIII. Branching
IX. Applications in medicine
X.
Applications in other fields
v
PREFACE
XI. A second bibliography on semi-Markov processes
It is interesting to quote that sections IV to X represent
a good sample of the main applications of semi-Markov processes
i.e. Economy, Insurance, Reliability, Simulation, Queueing, Branching,
Medicine (including survival data), Social Sciences, Language Model
ling, Seismic Risk, Analysis, Biology and Computer Science.
This strong interaction between recent theoretical results
presented in the first three sections and applications given above
was clearly pointed out by the participants.
I should like to thank all of them for their contribution and
their enthusiasm.
In particular, I should like to mention the strong support I
found by A.A. BOROVKOV (Institue of Mathematics, Novosibirsk),
E.~INLAR (Civil Engineering Department, Princeton University),
D.R.COX (Department of Mathematics, Imperial College, London),
V.S.KOROLYUK (Institute of Mathematics, Ukrainian Academy of Sciences,
Kiev), M.NEUTS (Department of Mathematical Sciences, University of
Delaware, Newark) and J.TEUGELS (Department of Mathematics, Katho
lieke Universiteit te Leuven). This last one provides me with in
valuable support.
Finally, I should mention the CADEPS (Centre d'Analyse des
Donnees et Processus Stochastiques, Ecole de Commerce Solvay,
Brussels) and the CEME (Centre d'Economie Mathematique et d'Econo
metrie, Brussels) which give me effective support by taking in hands
the local organization.
Here too, I have to retain particularly the strong help of
Mr ABIKHALIL (CADEPS) for the final preparation of this volume.
J. Janssen
CONTENTS
SECTION I. MARKOV ADDITIVE PROCESSES AND REGENERATIVE SYSTEMS
Some limit theorems for Markov additive processes . • • • • ••
P.Ney and E.Nummelin
Stationary regenerative processes
H.Kaspi and B.Maisonneuve
Asymptotic analysis of some non-homogeneous semi-Markov
processes
R.V.Benevento
SECTION II. SEMI-l~OV DECISION PROCESSES
Markov and semi-Markov decision models and optimal stopping
M.Schal
Markov decision drift processes
F.A.Van der Duyn Schouten
The functional equations of undiscounted denumerable
state rfurkov renewal programming • . • • • • •
E.Mann
SECTION III. ALGORITHMIC AND COMPUTER-ORIENTED APPROACH
Recursive moment formulas for regenerative simulation • • .•
P.W.Glynn and D.L.lglehart
3
13
23
39
63
79
99
Computation of the state probabilities in a class of semi-
regenerative queueing systems
H.Schellhaas
• • 111
viii
CONTENTS
The superposition of two PH-renewal processes • • • • . . . • • 131
M.Neuts and G.Latouche
SECTION IV. SEMI-MARKOV MODELS IN ECONOMY AND INSURANCE
Pension accumulation as a semi-Markov reward process,
with applications to pension reform • • • • • . • • . • 181
Y.Balcer and I.Sahin
The structure of a firm's optimal non-decreasing wage policy
when recruitment is a wage dependent Poisson process
N.H.Schager
SECTION V. SEMI-MARKOV PROCESSES AND RELIABILITY THEORY
Markov renewal processes in reliability analysis
V.S.Korolyuk
• 201
. 217
Deterioration processes • • • • • • • • • • • • . • • • • . • • 231
M.Abdel-Hameed
Stochastic processes with an embedded point process and
their application to system reliability analysis • . •• 253
P.Franken and A.Streller
SECTION VI. SIMULATION AND STATISTICS FOR SEMI-MARKOV PROCESSES
Semi-Markov models for manpower planning
S.McClean
. • • • . • • • • .
. 283
Statistical analysis of semi-Markov processes based on the
theory of counting processes • • • • •
• • • • • • 301
N.Keiding
SECTION VII. SEMI-MARKOV PROCESSES AND QUEUEING THEORY
Approximation of stochastic models
V.V.Kalashnikov
• 319
The method of renovating events and its applications
in queueing theory . • .
• • • • • • . • • • • • 337
S.G.Foss
On non-time homogeneity • • • . • • • • . • • • • • • . • . . • 351
H.Thorisson