Table of Contents
Copyright................................................................................................................................ 1
About the Authors.................................................................................................................. 2
Foreword................................................................................................................................ 2
Chapter 1. Introduction......................................................................................................... 10
Chapter 2. Fundamentals of Testing...................................................................................... 14
Section 2.1. Terms and Motivation................................................................................................................................................................................................ 15
Section 2.2. The Fundamental Test Process................................................................................................................................................................................. 26
Section 2.3. The Psychology of Testing......................................................................................................................................................................................... 40
Section 2.4. General Principles of Testing.................................................................................................................................................................................... 42
Section 2.5. Testing Ethics............................................................................................................................................................................................................ 44
Section 2.6. Summary................................................................................................................................................................................................................... 46
Chapter 3. Testing in the Software Lifecycle......................................................................... 48
Section 3.1. The General V-Model................................................................................................................................................................................................. 48
Section 3.2. Component Test......................................................................................................................................................................................................... 52
Section 3.3. Integration Test......................................................................................................................................................................................................... 59
Section 3.4. System Test................................................................................................................................................................................................................ 67
Section 3.5. Acceptance Test.......................................................................................................................................................................................................... 71
Section 3.6. Testing New Product Versions.................................................................................................................................................................................. 74
Section 3.7. Generic Types of Testing........................................................................................................................................................................................... 79
Section 3.8. Summary................................................................................................................................................................................................................... 85
Chapter 4. Static Testing....................................................................................................... 88
Section 4.1. Structured Group Examinations............................................................................................................................................................................... 88
Section 4.2. Static Analysis.......................................................................................................................................................................................................... 104
Section 4.3. Summary................................................................................................................................................................................................................... 111
Chapter 5. Dynamic Analysis – Test Design Techniques....................................................... 114
Section 5.1. Black Box Testing Techniques.................................................................................................................................................................................. 118
Section 5.2. White Box Testing Techniques................................................................................................................................................................................ 152
Section 5.3. Intuitive and Experience Based Test Case Determination..................................................................................................................................... 166
Section 5.4. Summary.................................................................................................................................................................................................................. 170
Chapter 6. Test Management............................................................................................... 174
Section 6.1. Test Organization..................................................................................................................................................................................................... 174
Section 6.2. Test Planning........................................................................................................................................................................................................... 179
Section 6.3. Cost and Economy Aspects...................................................................................................................................................................................... 185
Section 6.4. Definition of Test Strategy and Test Approach....................................................................................................................................................... 189
Section 6.5. Test Activity Management....................................................................................................................................................................................... 194
Section 6.6. Incident Management............................................................................................................................................................................................. 198
Section 6.7. Requirements to Configuration Management........................................................................................................................................................ 205
Section 6.8. Relevant Standards................................................................................................................................................................................................. 207
Section 6.9. Summary................................................................................................................................................................................................................. 208
Chapter 7. Test Tools........................................................................................................... 210
Section 7.1. Types of Test Tools................................................................................................................................................................................................... 210
Section 7.2. Selection and Introduction of Test Tools................................................................................................................................................................ 224
Section 7.3. Summary.................................................................................................................................................................................................................. 229
Part 1: Appendix.................................................................................................................. 232
Appendix A. Test Plan According to IEEE Standard 829-1998................................................................................................................................................. 232
Section A.1. Test Plan Identifier.............................................................................................................................................................................................. 232
Section A.2. Introduction........................................................................................................................................................................................................ 232
Section A.3. Test Objects or Items.......................................................................................................................................................................................... 233
Section A.4. Features to be tested........................................................................................................................................................................................... 233
Section A.5. Features not to be tested..................................................................................................................................................................................... 233
Section A.6. Test Approach or Strategy................................................................................................................................................................................... 233
Section A.7. Acceptance Criteria............................................................................................................................................................................................. 233
Section A.8. Suspension Criteria and Resumption Requirements......................................................................................................................................... 234
Section A.9. Test Deliverables................................................................................................................................................................................................. 234
Section A.10. Testing Tasks..................................................................................................................................................................................................... 235
Section A.11. Test Infrastructure and Environmental Needs................................................................................................................................................. 235
Section A.12. Responsibilities.................................................................................................................................................................................................. 235
Section A.13. Staffing and Training Needs.............................................................................................................................................................................. 236
Section A.14. Schedule............................................................................................................................................................................................................. 236
Section A.15. Risks and Contingencies.................................................................................................................................................................................... 236
Section A.16. Approvals........................................................................................................................................................................................................... 237
Section A.17. Glossary.............................................................................................................................................................................................................. 237
Appendix B. Important Information on the Curriculum and on the Certified Tester Exam.................................................................................................... 238
Appendix C. Exercises................................................................................................................................................................................................................. 240
Section C.1. Exercises to Chapter 2......................................................................................................................................................................................... 240
Section C.2. Exercises to Chapter 3......................................................................................................................................................................................... 240
Software Testing Foundations: A Study Guide for the Certified Tester Exam, 3rd Edition. Software Testing Foundations, Third Edition, ISBN: 9781933952789
Prepared for DianHui.Yang@fmr.com, Frank Yang
© 2013 Safari Books Online, LLC. This download file is made available for personal use only and is subject to the Terms of Service. Any other use requires prior written consent from the
copyright owner. Unauthorized use, reproduction and/or distribution are strictly prohibited and violate applicable laws. All rights reserved.
Section C.3. Exercises to Chapter 4......................................................................................................................................................................................... 241
Section C.4. Exercises to Chapter 5......................................................................................................................................................................................... 241
Section C.5. Exercises to Chapter 6......................................................................................................................................................................................... 242
Section C.6. Exercises to Chapter 7......................................................................................................................................................................................... 242
Appendix D. Glossary.................................................................................................................................................................................................................. 244
Literature..................................................................................................................................................................................................................................... 278
Bibliodiv................................................................................................................................................................................................................................... 286
Further recommended literature............................................................................................................................................................................................ 286
Standards................................................................................................................................................................................................................................. 286
WWW-Pages............................................................................................................................................................................................................................ 286
Further useful WWW-pages.................................................................................................................................................................................................... 286
bvdindexIndex.................................................................................................................... 286
Software Testing Foundations: A Study Guide for the Certified Tester Exam, 3rd Edition. Software Testing Foundations, Third Edition, ISBN: 9781933952789
Prepared for DianHui.Yang@fmr.com, Frank Yang
© 2013 Safari Books Online, LLC. This download file is made available for personal use only and is subject to the Terms of Service. Any other use requires prior written consent from the
copyright owner. Unauthorized use, reproduction and/or distribution are strictly prohibited and violate applicable laws. All rights reserved.
Software Testing Foundations: A Study Guide for the Certified Tester Exam, 3rd Edition. Software Testing Foundations, Third Edition, ISBN: 9781933952789
Prepared for DianHui.Yang@fmr.com, Frank Yang
© 2013 Safari Books Online, LLC. This download file is made available for personal use only and is subject to the Terms of Service. Any other use requires prior written consent from the
copyright owner. Unauthorized use, reproduction and/or distribution are strictly prohibited and violate applicable laws. All rights reserved.
ivAndreas Spillner andreas.spillner@hs-bremen.de Tilo Linz tilo.linz@imbus.de Hans Schaefer hans.schaefer@ieee.org Editor: Dr. Michael BarabasCopyeditor: Judy FlynnLayout and Type: Birgit Bäuerlein/Nadine Thiele Cover Design: Helmut Kraus, www.exclam.de Printer: Courier CorporationPrinted in the USAISBN: 978-1-933952-78-93rd Edition © 2011 by Rocky Nook Inc. 26 West Mission Street Ste 3Santa Barbara, CA 93101 www.rockynook.comThis 3rd English book edition conforms to the 4th German edition “Basiswissen Softwaretest – Aus- und Weiterbildung zum Certified Tester – Foundation Level nach ISTQB-Standard” (dpunkt.verlag GmbH, ISBN 3-89864-358-1), which was published in May 2010. Library of Congress Cataloging-in-Publication DataSpillner, Andreas. Software testing foundations : a study guide for the certified tester exam / Andreas Spillner, Tilo Linz, Hans Schaefer. -- 3rd ed. p. cm. ISBN 978-1-933952-78-9 (soft cover : alk. paper)1. Computer software--Testing. 2. Computer software--Verification. 3. Computer software--Evaluation. I. Linz, Tilo. II. Schaefer, Hans, 1952- III. Title. QA76.76.T48S66 2011 005.1'4--dc22 2010043374Distributed by O’Reilly Media1005 Gravenstein Highway NorthSebastopol, CA 95472All product names and services identified throughout this book are trademarks or registered trademarks of their respective companies. They are used throughout this book in editorial fashion only and for the benefit of such companies. No such uses, or the use of any trade name, is intended to convey endorsement or other affiliation with the book. No part of the material protected by this copyright notice may be reproduced or utilized in any form, electronic or mechanical, including photocopying, recording, or bay any information storage and retrieval system, without written permission from the copyright owner.
Licensed by
Frank Yang
3287325
Software Testing Foundations: A Study Guide for the Certified Tester Exam, 3rd Edition. Software Testing Foundations, Third Edition, ISBN: 9781933952789
Prepared for DianHui.Yang@fmr.com, Frank Yang
© 2013 Safari Books Online, LLC. This download file is made available for personal use only and is subject to the Terms of Service. Any other use requires prior written consent from the
copyright owner. Unauthorized use, reproduction and/or distribution are strictly prohibited and violate applicable laws. All rights reserved.
Foreword vForewordWe really do mean “Foundations”The International Software Testing Qualifications Board (ISTQB) was offi-cially founded in Edinburgh in November 2002. A lot can happen in thespace of eight years in the IT world—new developments find their way intoeveryday use and new versions of established processes become available.In spite of this constant change, information technology is built on funda-mental know-how that generally remains constant. From the outset, wehave always taken the “Foundations” part of our title seriously, and weavoid addressing aspects of IT that are not yet proven. Specialty skills suchas Web application or embedded system testing are not IT basics. Wherenecessary, we reference current literature on specialist topics.Knowledge is the cornerstone of ITThe major advantage of true grass-roots knowledge is that it can beapplied to many different disciplines. The ISTQB Foundation Level certi-fication is in worldwide demand, clearly demonstrated by the strength ofthe ISTQB's membership, which has grown continually from a handful ofmembers in 2002 and today counts 47 Testing Boards around the world. Acurrent list of the individual country websites can be found on the ISTQBWorldwide tab at www.istqb.org. It is no exaggeration to say that theISTQB and its Certified Testers are a globally recognized force that con-tributes significantly to worldwide software testing dependability.Thanks are due …This success has only been possible due to the outstanding coopera-tion between all those involved. We would like to thank everyone atISTQB, without whom the Certified Tester scheme couldn’t have achievedthe level of global acceptance it enjoys today.What’s new in this edition?So why are we publishing a new edition of this book? We have cor-rected a number of errors and inconsistencies, and we would like to thankthe readers who drew our attention to some of these. We have alsoincreased the precision of some definitions and updated others to matchthe latest version of the ISTQB glossary. This version of the book also con-forms to amendments and additions that have been made to the ISTQB’s2010 syllabus, and we have updated the bibliography to include new refer-
Software Testing Foundations: A Study Guide for the Certified Tester Exam, 3rd Edition. Software Testing Foundations, Third Edition, ISBN: 9781933952789
Prepared for DianHui.Yang@fmr.com, Frank Yang
© 2013 Safari Books Online, LLC. This download file is made available for personal use only and is subject to the Terms of Service. Any other use requires prior written consent from the
copyright owner. Unauthorized use, reproduction and/or distribution are strictly prohibited and violate applicable laws. All rights reserved.
viForewordence books and standards. We have updated the quoted URLs where nec-essary. Our new website Advanced Level books Additionally, we have launched a new website at http://www.soft-waretest-knowledge.net/, where we will regularly post any changes that aremade to the syllabus or the glossary after this book goes to print. The web-site will also include any corrections that we may have to make to the booktext. Our web pages include references to books that cover the AdvancedLevel certification syllabus. We wish our readers every success in turning the theory contained inthis book into practice. If you are using the book to prepare for the Certi-fied Tester Foundation Level examination, we wish you all the best on thebig day.Andreas Spillner, Tilo Linz and Hans SchaeferBremen, Möhrendorf, and Valestrandsfossen, December 2010
Software Testing Foundations: A Study Guide for the Certified Tester Exam, 3rd Edition. Software Testing Foundations, Third Edition, ISBN: 9781933952789
Prepared for DianHui.Yang@fmr.com, Frank Yang
© 2013 Safari Books Online, LLC. This download file is made available for personal use only and is subject to the Terms of Service. Any other use requires prior written consent from the
copyright owner. Unauthorized use, reproduction and/or distribution are strictly prohibited and violate applicable laws. All rights reserved.
Contents viiContents1Introduction 12Fundamentals of Testing 52.1Terms and Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.1.1Error and Bug Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.1.2Testing Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.1.3Software Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.1.4Test Effort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.2The Fundamental Test Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172.2.1Test Planning and Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.2.2Test Analysis and Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212.2.3Test Implementation and Execution . . . . . . . . . . . . . . . . . . . . . . . 252.2.4Evaluation of Exit Criteria and Reporting . . . . . . . . . . . . . . . . . . 272.2.5Test Closure Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302.3The Psychology of Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312.4General Principles of Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332.5Testing Ethics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352.6Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373Testing in the Software Lifecycle 393.1The General V-Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393.2Component Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433.2.1Explanation of Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433.2.2Test Basis and Test Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433.2.3Test Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443.2.4Test Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463.2.5Test Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Software Testing Foundations: A Study Guide for the Certified Tester Exam, 3rd Edition. Software Testing Foundations, Third Edition, ISBN: 9781933952789
Prepared for DianHui.Yang@fmr.com, Frank Yang
© 2013 Safari Books Online, LLC. This download file is made available for personal use only and is subject to the Terms of Service. Any other use requires prior written consent from the
copyright owner. Unauthorized use, reproduction and/or distribution are strictly prohibited and violate applicable laws. All rights reserved.
viiiContents3.3Integration Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503.3.1Explanation of Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503.3.2Test Basis and Test Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523.3.3Test Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 533.3.4Test Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 543.3.5Integration Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 553.4System Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 583.4.1Explanation of Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 583.4.2Test Basis, Test Objects and Test Environment . . . . . . . . . . . . . 593.4.3Test Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 613.4.4Problems in System Test Practice . . . . . . . . . . . . . . . . . . . . . . . . . . 613.5Acceptance Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 623.5.1Testing for Acceptance According to the Contract . . . . . . . . . 633.5.2Testing for User Acceptance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 643.5.3Operational (Acceptance) Testing . . . . . . . . . . . . . . . . . . . . . . . . . 643.5.4Field Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 643.6Testing New Product Versions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 653.6.1Software Maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 653.6.2Release Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 683.6.3Testing in Incremental Development . . . . . . . . . . . . . . . . . . . . . . 693.7Generic Types of Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 703.7.1Functional Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 703.7.2Nonfunctional Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 723.7.3Testing of Software Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 743.7.4Testing Related to Changes and Regression Testing . . . . . . . 753.8Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 764Static Testing 794.1Structured Group Examinations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 794.1.1Foundations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 794.1.2Reviews . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 804.1.3The General Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 824.1.4Roles and Responsibilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 864.1.5Types of Reviews . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88