Handbook of Parallel Computing
and Statistics
© 2006 by Taylor & Francis Group, LLC
STATISTICS: Textbooks and Monographs
D. B. Owen
Founding Editor, 1972–1991
Associate Editors
Statistical Computing/
Nonparametric Statistics
Professor William R. Schucany
Southern Methodist University
Probability
Professor Marcel F. Neuts
University of Arizona
Multivariate Analysis
Professor Anant M. Kshirsagar
University of Michigan
Quality Control/Reliability
Professor Edward G. Schilling
Rochester Institute of
Technology
Editorial Board
Applied Probability
Dr. Paul R. Garvey
The MITRE Corporation
Statistical Process Improvement
Professor G. Geoffrey Vining
Virginia Polytechnic Institute
Economic Statistics
Professor David E. A. Giles
University of Victoria
Stochastic Processes
Professor V. Lakshmikantham
Florida Institute of Technology
Survey Sampling
Professor Lynne Stokes
Southern Methodist University
Time Series
Sastry G. Pantula
North Carolina State University
Experimental Designs
Mr. Thomas B. Barker
Rochester Institute of
Technology
Multivariate Analysis
Professor Subir Ghosh
University of California,
Riverside
Statistical Distributions
Professor N. Balakrishnan
McMaster University
© 2006 by Taylor & Francis Group, LLC
STATISTICS: Textbooks and Monographs
Recent Titles
Statistical Process Monitoring and Optimization, edited by Sung H. Park and
G. Geoffrey Vining
Statistics for the 21st Century: Methodologies for Applications of the Future, edited by C.
R. Rao and Gábor J. Székely
Probability and Statistical Inference, Nitis Mukhopadhyay
Handbook of Stochastic Analysis and Applications, edited by D. Kannan and
V. Lakshmikantham
Testing for Normality, Henry C. Thode, Jr.
Handbook of Applied Econometrics and Statistical Inference, edited by Aman Ullah, Alan
T. K. Wan, and Anoop Chaturvedi
Visualizing Statistical Models and Concepts, R. W. Farebrother and Michaël Schyns
Financial and Actuarial Statistics: An Introduction, Dale S. Borowiak
Nonparametric Statistical Inference, Fourth Edition, Revised and Expanded, Jean Dickinson
Gibbons and Subhabrata Chakraborti
Computer-Aided Econometrics, edited by David E.A. Giles
The EM Algorithm and Related Statistical Models, edited by Michiko Watanabe and
Kazunori Yamaguchi
Multivariate Statistical Analysis, Second Edition, Revised and Expanded, Narayan C. Giri
Computational Methods in Statistics and Econometrics, Hisashi Tanizaki
Applied Sequential Methodologies: Real-World Examples with Data Analysis, edited by Nitis
Mukhopadhyay, Sujay Datta, and Saibal Chattopadhyay
Handbook of Beta Distribution and Its Applications, edited by Arjun K. Gupta and Saralees
Nadarajah
Item Response Theory: Parameter Estimation Techniques, Second Edition, edited by Frank
B. Baker and Seock-Ho Kim
Statistical Methods in Computer Security, edited by William W. S. Chen
Elementary Statistical Quality Control, Second Edition, John T. Burr
Data Analysis of Asymmetric Structures, Takayuki Saito and Hiroshi Yadohisa
Mathematical Statistics with Applications, Asha Seth Kapadia, Wenyaw Chan, and Lemuel Moyé
Advances on Models, Characterizations and Applications, N. Balakrishnan, I. G. Bairamov,
and O. L. Gebizlioglu
Survey Sampling: Theory and Methods, Second Edition, Arijit Chaudhuri and Horst Stenger
Statistical Design of Experiments with Engineering Applications, Kamel Rekab and
Muzaffar Shaikh
Quality By Experimental Design, Third Edition, Thomas B. Barker
Handbook of Parallel Computing and Statistics, Erricos John Kontoghiorghes
Statistical Inference Based on Divergence Measures, Leandro Pardo
A Kalman Filter Primer, Randy Eubank
© 2006 by Taylor & Francis Group, LLC
Handbook of Parallel Computing
and Statistics
Edited by
Erricos John Kontoghiorghes
University of Cyprus and
Birkbeck College, University of London, UK
© 2006 by Taylor & Francis Group, LLC
Boca Raton London New York
DK2384_Discl.fm Page 1 Friday, November 11, 2005 5:42 PM
Published in 2006 by
Chapman & Hall/CRC
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2006 by Taylor & Francis Group, LLC
Chapman & Hall/CRC is an imprint of Taylor & Francis Group
No claim to original U.S. Government works
Printed in the United States of America on acid-free paper
10 9 8 7 6 5 4 3 2 1
International Standard Book Number-10: 0-8247-4067-X (Hardcover)
International Standard Book Number-13: 978-0-8247-4067-2 (Hardcover)
This book contains information obtained from authentic and highly regarded sources. Reprinted material is quoted with
permission, and sources are indicated. A wide variety of references are listed. Reasonable efforts have been made to publish
reliable data and information, but the author and the publisher cannot assume responsibility for the validity of all materials
or for the consequences of their use.
No part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or
other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information
storage or retrieval system, without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright.com
(http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC) 222 Rosewood Drive, Danvers, MA
01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For
organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for
identification and explanation without intent to infringe.
Library of Congress Cataloging-in-Publication Data
Catalog record is available from the Library of Congress
Taylor & Francis Group
is the Academic Division of Informa plc.
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
© 2006 by Taylor & Francis Group, LLC
i
i
Erricos John Kontoghiorghes: Handbook of Parallel Computing and Statistics DK2384 prelims 2005/11/16 10:12
i
i
Dedication
This book is dedicated to Laurence, Louisa, Ioanna, and Emily
© 2006 by Taylor & Francis Group, LLC
i
i
i
i
i
i
Erricos John Kontoghiorghes: Handbook of Parallel Computing and Statistics DK2384 prelims 2005/11/16 10:12
i
i
Preface
Over the last decades a wealth of parallel algorithms has been discovered for solving a wide range
of problems that arise in diverse applications areas. Effort has been concentrated mainly on the
solution of large scale industrial and engineering problems. Although some of these application
areas, such as signal processing and pattern recognition, involve significant statistical computing
components, the development of parallel algorithms for general use in statistics and econometrics
remains comparatively neglected. This is, to some extent, due to a lack of strong interaction between
the parallel computing and statistical communities.
A number of current parallel numerical libraries provide subroutines that are useful to statisti-
cians. For example, most of them offer routines to solve constrained least squares problems and
matrix problems that arise in statistical modelling and estimation. However, these routines have
been constructed as numerical tools for general use and are unsuitable for the efficient solution of
statistical problems that exhibit special properties and characteristics. The design of specifically
targeted parallel numerical libraries and tools to facilitate the solution of computationally inten-
sive statistical problems requires close collaboration between statisticians and parallel computing
experts [1–3].
The aim of this handbook is twofold: first, to provide an overview of the state-of-the-art in par-
allel algorithms and processing from a statistical computing standpoint; and, second, to contribute
toward the development and deepening of research in the interface between parallel and statistical
computation.
REFERENCES
[1] E.J. Kontoghiorghes. Parallel Algorithms for Linear Models: Numerical Methods and Estima-
tion Problems. Advances in Computational Economics, Vol. 15. Kluwer Academic Publishers,
Boston, MA, 2000.
[2] E.J. Kontoghiorghes. Parallel processing and statistics. Special issue, Computational Statistics
& Data Analysis, 31(4), 373–516, 1999.
[3] E.J. Kontoghiorghes, A. Nagurney, and B. Rustem. Parallel Computing in economics, finance,
and decision-making. Special issue, Parallel Computing, 26(5), 507–676, 2000.
© 2006 by Taylor & Francis Group, LLC
i
i
i
i
i
i
Erricos John Kontoghiorghes: Handbook of Parallel Computing and Statistics DK2384 prelims 2005/11/16 10:12
i
i
Editor
Erricos John Kontoghiorghes received a B.Sc. and Ph.D. from Queen Mary College, University of
London, United Kingdom. He is a faculty member of the Department of Public and Business Ad-
ministration at University of Cyprus and holds a visiting professorship at Birkbeck College, Univer-
sity of London, United Kingdom. Previously, he held faculty positions at City University Business
School, London, United Kingdom and University of Neuchatel, Switzerland, and held various
visiting appointments at INRIA-IRISA, France.
Dr. Kontoghiorghes has published over 50 technical papers in journals and books in the interface
research area of parallel computing, statistics, econometrics, and numerical linear algebra. He has
authored or edited 7 books, and edited 11 special issues of journals. He is a co-editor of the Journal
of Computational Statistics & Data Analysis (Elsevier), editor-in-chief of the Handbook Series of
Computing and Statistics with Applications (Elsevier), and a member of the editing committees
of various journals. He was elected vice-president of the International Association for Statistical
Computing (IASC) in April 2005.
© 2006 by Taylor & Francis Group, LLC
i
i
i
i