8560 Prelims (i-xviii) 10/10/02 1:46 pm Page i
Testing Embedded Software
8560 Prelims (i-xviii) 10/10/02 1:46 pm Page ii
8560 Prelims (i-xviii) 10/10/02 1:46 pm Page iii
Bart Broekman and
Edwin Notenboom
Testing
Embedded
Software
An imprint of PEARSON EDUCATION
London · Boston · Indianapolis · New York · Mexico City · Toronto · Sydney · Tokyo · Singapore
Hong Kong · Cape Town · New Delhi · Madrid · Paris · Amsterdam · Munich · Milan · Stockholm
8560 Prelims (i-xviii) 10/10/02 1:46 pm Page iv
London Office
128 Long Acre
London WC2E 9AN
Tel: +44 (0)20 7447 2000
Fax: +44 (0)20 7447 2170
PEARSON EDUCATION LIMITED
Head Office
Edinburgh Gate
Harlow CM20 2JE
Tel: +44 (0)1279 623623
Fax: +44 (0)1279 431059
Website: www.it-minds.com
www.aw.professional.com
First Published in Great Britain in 2003
© Sogeti Nederland, 2003
The right of Bart Broekman and Edwin Notenboom to be identified as
the Authors of this work has been asserted by them in accordance with
the Copyright, Designs and Patents Act 1988.
ISBN 0 321 15986 1
British Library Cataloguing in Publication Data
A CIP catalogue record for this book is available from the British Library.
Library of Congress Cataloging in Publication Data
Applied for.
All rights reserved. No part of this publication may be reproduced, stored
in a retrieval system, or transmitted in any form or by any means, electronic,
mechanical, photocopying, recording, or otherwise without either the prior
written permission of the Publishers or a licence permitting restricted copying
in the United Kingdom issued by the Copyright Licensing Agency Ltd,
90 Tottenham Court Road, London W1T 4LP. This book may not be lent,
resold, hired out or otherwise disposed of by way of trade in any form of binding
or cover other than that in which it is published, without the prior consent
of the Publishers.
Approval has been obtained from ETAS GmbH, Stuttgart, to use the pictures that they provided.
TMap® is a registered trademark of Sogeti Nederland BV.
10 9 8 7 6 5 4 3 2 1
Typeset by Pantek Arts Ltd, Maidstone, Kent.
Printed and bound in Great Britain by Biddles Ltd, Guildford and King’s Lynn.
The Publishers’ policy is to use paper manufactured from sustainable forests.
8560 Prelims (i-xviii) 10/10/02 1:46 pm Page v
Contents
Foreword
Preface
Acknowledgments
Part I Introduction
1
Fundamentals
Aims of testing
1.1
1.2 What is an embedded system?
1.3
Approach to the testing of embedded systems
2
The TEmb method
2.1
2.2
2.3 Mechanism for assembling the dedicated test approach
Overview
TEmb generic
Part II Lifecycle
3 Multiple V-model
3.1
3.2
3.3
Introduction
Test activities in the multiple Vs
The nested multiple V-model
4 Master test planning
4.1
4.2
Elements of master test planning
Activities
x
xiii
xvi
xix
3
3
5
6
7
7
10
15
21
25
25
27
29
33
33
37
8560 Prelims (i-xviii) 10/10/02 1:46 pm Page vi
vi
Testing Embedded Software
5
6
Testing by developers
5.1
5.2
5.3
Introduction
Integration approach
Lifecycle
Testing by an independent test team
6.1
6.2
6.3
6.4
6.5
6.6
Introduction
Planning and control phase
Preparation phase
Specification phase
Execution phase
Completion phase
Part III Techniques
7
Risk-based test strategy
7.1
7.2
7.3
7.4
7.5
7.6
Introduction
Risk assessment
Strategy in master test planning
Strategy for a test level
Strategy changes during the test process
Strategy for maintenance testing
8
9
Testability review
8.1
8.2
Introduction
Procedure
Inspections
9.1
9.2
Introduction
Procedure
10 Safety analysis
10.1 Introduction
10.2 Safety analysis techniques
10.3 Safety analysis lifecycle
11 Test design techniques
11.1 Overview
11.2 State transition testing
45
45
46
50
55
55
55
64
66
69
72
75
79
79
80
82
85
90
91
95
95
95
99
99
100
103
103
104
109
113
113
121
8560 Prelims (i-xviii) 10/10/02 1:46 pm Page vii
Contents
vii
11.3 Control flow test
11.4 Elementary comparison test
11.5 Classification-tree method
11.6 Evolutionary algorithms
11.7 Statistical usage testing
11.8 Rare event testing
11.9 Mutation analysis
12 Checklists
12.1 Introduction
12.2 Checklists for quality characteristics
12.3 General checklist for high-level testing
12.4 General checklist for low-level testing
12.5 Test design techniques checklist
12.6 Checklists concerning the test process
Part IV Infrastructure
13 Embedded software test environments
13.1 Introduction
13.2 First stage: simulation
13.3 Second stage: prototyping
13.4 Third stage: pre-production
13.5 Post-development stage
14 Tools
14.1 Introduction
14.2 Categorization of test tools
15 Test automation
15.1 Introduction
15.2 The technique of test automation
15.3 Implementing test automation
16 Mixed signals
Mirko Conrad and Eric Sax
16.1 Introduction
16.2 Stimuli description techniques
16.3 Measurement and analysis techniques
134
138
144
151
158
165
166
169
169
169
175
176
177
178
189
193
193
195
199
205
207
209
209
210
217
217
218
222
229
229
234
245
8560 Prelims (i-xviii) 10/10/02 1:46 pm Page viii
viii
Testing Embedded Software
Part V Organization
17 Test roles
17.1 General skills
17.2 Specific test roles
18 Human resource management
18.1 Staff
18.2 Training
18.3 Career perspectives
19 Organization structure
19.1 Test organization
19.2 Communication structures
20 Test control
20.1 Control of the test process
20.2 Control of the test infrastructure
20.3 Control of the test deliverables
Part VI Appendices
Appendix A Risk classification
Appendix B Statecharts
B.1
B.2
B.3
B.4
B.5
B.6
States
Events
Transitions
Actions and activities
Execution order
Nested states
Appendix C Blueprint of an automated test suite
C.1
C.2
C.3
C.4
C.5
C.6
Test data
Start
Planner
Reader
Translator
Test actions
251
255
255
256
265
265
267
268
273
273
277
279
279
284
286
291
293
295
295
296
297
297
298
299
301
301
302
302
303
304
304