Foreword
Acknowledgments
Contents
Contributors
About the Editors
1 A Preliminary Taxonomy for Machine Learning in VLSI CAD
1.1 Machine Learning Taxonomy
1.1.1 Unsupervised, Supervised, and Semisupervised Learning
1.1.2 Parametric and Nonparametric Methods
1.1.3 Discriminative Versus Generative Methods
1.2 VLSI CAD Abstraction Levels
1.3 Organization of This Book
1.3.1 Machine Learning for Lithography and Physical Design
1.3.1.1 Shiely—Compact Lithographic Process Models
1.3.1.2 Shim et al.—Mask Synthesis
1.3.1.3 Lin and Pan—Physical Verification, Mask Synthesis, and Physical Design
1.3.2 Machine Learning for Manufacturing, Yield, and Reliability
1.3.2.1 Xanthopoulos et al.—Gaussian Process for Wafer-Level Correlations
1.3.2.2 Chen and Boning—Yield Enhancement
1.3.2.3 Tao et al.—Virtual Probe
1.3.2.4 Xiong et al.—Chip Testing
1.3.2.5 Vijayan et al.—Aging Analysis
1.3.3 Machine Learning for Failure Modeling
1.3.3.1 Singhee—Extreme Statistics in Memories
1.3.3.2 Kanj et al.—Fast Statistical Analysis Using Logistic Regression
1.3.3.3 Tao et al.—Fast Statistical Analysis of Rare Circuit Failures
1.3.3.4 Wang—Learning from Limited Data
1.3.4 Machine Learning for Analog Design
1.3.4.1 Tao et al.—Bayesian Model Fusion
1.3.4.2 Lin et al.—Sparse Relevance Kernel Machine
1.3.4.3 Singhee—Projection Pursuit with SiLVR
1.3.4.4 Torun et al.—Integrated Voltage Regulator Optimization and Uncertainty Quantification
1.3.5 Machine Learning for System Design and Optimization
1.3.5.1 Ziegler et al.—SynTunSys
1.3.5.2 Karn and Elfadel—Multicore Power and Thermal Proxies
1.3.5.3 Vasudevan et al.—GoldMine for RTL Assertion Generation
1.3.5.4 Hanif et al.—Machine Learning Architectures and Hardware Design
1.3.6 Other Work and Outlook
References
Part I Machine Learning for Lithography and Physical Design
2 Machine Learning for Compact Lithographic Process Models
2.1 Introduction
2.2 The Lithographic Patterning Process
2.2.1 Importance of Lithographic Patterning Process to the Economics of Computing
2.2.2 Representation of the Lithographic Patterning Process
2.2.2.1 Mask Transfer Function
2.2.2.2 Imaging Transfer Function
2.2.2.3 Resist Transfer Function
2.2.2.4 Etch Transfer Function
2.2.3 Summary
2.3 Machine Learning of Compact Process Models
2.3.1 The Compact Process Model Machine Learning Problem Statement
2.3.1.1 The Compact Process Model Task
2.3.1.2 The CPM Training Experience
2.3.1.3 CPM Performance Metrics
2.3.1.4 Summary of CPM Problem Statement
2.3.2 Supervised Learning of a CPM
2.3.2.1 CPM Model Form
2.3.2.2 CPM Supervised Learning Dataset
2.3.2.3 CPM Supervised Learning Cost Function
2.3.2.4 CPM Supervised Learning Optimization Algorithm
2.4 Neural Network Compact Patterning Models
2.4.1 Neural Network Mask Transfer Function
2.4.2 Neural Network Image Transfer Function
2.4.2.1 Neural Network Mask Diffraction Model
2.4.2.2 Summary of Neural Network Image Transforms
2.4.3 Neural Network Resist Transfer Function
2.4.3.1 Motivation for Neural Network Resist Transfer Functions
2.4.3.2 Earliest Neural Network Resist Model
2.4.3.3 Neural Network Resist Model That Maps to a Contour
2.4.3.4 Neural Networks of Two- and Three-Dimensional Resist Models
2.4.3.5 Convolutional Neural Network Resist Model
2.4.3.6 Summary of Neural Network Resist Transfer Functions
2.4.4 Neural Network Etch Transfer Functions
2.4.5 Summary of Neural Network Compact Patterning Models
2.5 Conclusions
References
3 Machine Learning for Mask Synthesis
3.1 Introduction
3.2 Machine Learning-Guided OPC
3.2.1 Prior Works
3.2.2 ML-OPC with PFT Signals as Model Parameters
3.2.2.1 PFT Signal
3.2.2.2 MLP Construction
3.2.3 Experiments
3.3 Machine Learning-Guided EPC
3.3.1 Preliminaries
3.3.1.1 Etch Bias and EPC
3.3.1.2 Rule- and Model-Based EPC
3.3.2 ML-EPC
3.3.2.1 Preparation of Training Segments
3.3.2.2 Extracting Parameters
3.3.2.3 Construction of Etch Bias Model
3.3.2.4 EPC Algorithm
3.3.3 Experiments
3.3.3.1 Assessment of Accuracy
3.3.3.2 Comparison of Two MLP Types
3.3.3.3 Sampling of Training Segments
3.3.3.4 Changing the Parameter Sets
3.3.3.5 Effect of Cross-validation
3.3.3.6 Assessment of ML-EPC
3.4 Conclusions
References
4 Machine Learning in Physical Verification, Mask Synthesis, and Physical Design
4.1 Introduction
4.2 Machine Learning in Physical Verification
4.2.1 Layout Feature Extraction and Encoding
4.2.2 Machine Learning Models for Hotspot Detection
4.3 Machine Learning in Mask Synthesis
4.3.1 Mask Synthesis Flow
4.3.2 Machine Learning for Sub-resolution Assist Features
4.3.3 Machine Learning for Optical Proximity Correction
4.4 Machine Learning in Physical Design
4.4.1 Machine Learning for Datapath Placement
4.4.2 Machine Learning for Routability-Driven Placement
4.4.3 Machine Learning for Clock Optimization
4.4.4 Machine Learning for Lithography Friendly Routing
4.5 Conclusions
References
Part II Machine Learning for Manufacturing, Yield, and Reliability
5 Gaussian Process-Based Wafer-Level Correlation Modeling and Its Applications
5.1 Introduction
5.1.1 Types of Wafer-Level Test Measurements
5.1.2 Wafer-Level Statistical Correlations
5.2 Gaussian Process-Based Regression Models
5.2.1 Modeling Covariance with Kernel Functions
5.2.2 Training and Prediction
5.2.3 Regularization
5.2.4 Modeling Radial Variation
5.2.5 Model Selection and Adaptation of Hyperparameters
5.2.6 Handling Discontinuous Effect in GP Modeling
5.2.7 Progressive Sampling (GP-PS)
5.2.8 Spatiotemporal Feature Inclusion (GP-ST)
5.2.9 Spatiotemporal GP w. Progressive Sampling (GP-ST-PS)
5.2.10 Considerations for Production Test Deployment
5.2.11 High-Volume Manufacturing Yield Estimation
5.2.11.1 Histogram with Random Sampling
5.2.11.2 Histogram with GP-ST-PS
5.2.11.3 Kernel Density Estimation
5.2.12 Prediction Evaluation
5.3 Applications
5.3.1 Spatial Correlation Modeling of E-Test Measurement
5.3.1.1 Virtual Probe
5.3.1.2 Proposed Method: Gaussian Process Models
5.3.1.3 Comparison to Virtual Probe
5.3.2 Spatial Correlation Modeling of Probe-Test Specification Measurements
5.3.2.1 Stationarity Verification of Discontinuous Spatial Patterns
5.3.2.2 Prediction Errors Using the Proposed Approach
5.3.2.3 Comparison to Existing Approaches
5.3.2.4 Test Escape and Yield Loss Improvement
5.3.3 IC Laser Trimming Speedup Through Spatial Correlation Modeling
5.3.3.1 Length-Based, Original Target Prediction
5.3.3.2 Rate-Based, Optimized Target Prediction
5.3.4 HVM Yield Estimation
5.3.4.1 Accuracy Improvement of Enhanced Model
5.3.4.2 Yield Estimation Results
5.4 Conclusions
Appendix 1: Proof of Positive Semi-Definite for Covariance Matrix
Appendix 2: Marginal and Conditional Distribution of Multivariate Normal Distribution
Appendix 3: Summary of Commonly Used Kernel Functions
References
6 Machine Learning Approaches for IC Manufacturing Yield Enhancement
6.1 Introduction
6.1.1 Challenge One: Imbalanced Classification
6.1.2 Challenge Two: Concept Drift
6.1.3 Application
6.2 Background of the Manufacturing Process
6.3 Preliminaries
6.3.1 Evaluation Metric: Confusion Matrix
6.3.2 Evaluation Metric: ROC Curves
6.3.3 Mathematical Formulation
6.4 Learning Models
6.4.1 Imbalanced Classification and Batch RUSBoost Learning
6.4.2 Online RUSBoost Learning
6.4.3 Incremental Learning for Concept Drift and Class Imbalance
6.5 Experimental Results
6.5.1 RUSBoost on Imbalanced Dataset
6.5.2 The Effectiveness of Online Learning
6.5.3 Incremental Learning with Concept Drift
6.6 Conclusions
References
7 Efficient Process Variation Characterization by Virtual Probe
7.1 Introduction
7.2 Virtual Probe
7.2.1 Mathematical Formulation
7.2.2 Maximum A Posteriori (MAP) Estimation
7.2.3 Accuracy of MAP Estimation
7.3 Implementation Details
7.3.1 Normalization
7.3.2 Linear Programming
7.3.3 Latin Hypercube Sampling
7.3.4 DCT Coefficient Preselection
7.3.5 Summary
7.4 Applications of Virtual Probe
7.4.1 Wafer-Level Silicon Characterization
7.4.2 Chip-Level Silicon Characterization
7.4.3 Beyond Silicon Characterization
7.5 Numerical Experiments
7.5.1 Flush Delay Measurement Data
7.5.1.1 Spatial Sample Generation
7.5.1.2 Spatial Variation Prediction
7.5.2 Leakage Current Measurement Data
7.5.3 Ring Oscillator Period Measurement Data
7.6 Conclusions
References
8 Machine Learning for VLSI Chip Testing and Semiconductor Manufacturing Process Monitoring and Improvement
8.1 Introduction
8.2 Background
8.3 Machine Learning for Chip Testing and Yield Optimization
8.3.1 Robust Spatial Correlation Extraction
8.3.1.1 Problem Formulations
8.3.1.2 Extraction Algorithms
8.3.1.3 Experimental Results
8.3.2 Statistical Chip Testing and Yield Optimization
8.3.2.1 Statistical Test Margin Computation
8.4 Hierarchical Multitask Learning for Wafer Quality Prediction
8.4.1 Problem Formulation
8.4.2 HEAR Algorithm
8.4.3 Experimental Results
8.5 Co-clustering Structural Temporal Data from Semiconductor Manufacturing
8.5.1 Problem Formulation
8.5.2 C-Struts Algorithm
8.5.3 Experimental Results
8.6 Conclusions
References
9 Machine Learning-Based Aging Analysis
9.1 Introduction
9.2 Negative Bias Temperature Instability
9.3 Related Prior Work
9.4 Proposed Technique
9.5 Offline Correlation Analysis and Prediction Model Generation
9.5.1 Aging-Induced Delay Degradation and SP Extraction
9.5.2 Predictor Training Using Support-Vector Machines
9.5.3 Representative Flip-Flop Selection (Space Sampling)
9.5.3.1 Correlation-Based Flip-Flop Selection
9.5.3.2 Fan-In Cone-Based Flip-Flop Selection
9.5.4 Time Complexity of Flip-Flop Selection Methods
9.5.5 Time Sampling
9.6 Runtime Stress Monitoring
9.7 Results
9.7.1 Experimental Setup
9.7.2 SVM Training and Validation
9.7.3 Evaluation of Prediction Accuracy
9.7.3.1 Joint Space–Time Sampling
9.7.3.2 Step-by-Step Correlation
9.7.4 Validation of Time-Sampling Hardware Design
9.7.5 Overheads
9.7.5.1 Performance Overhead
9.7.5.2 Area and Power Overhead
9.7.5.3 Overhead at Design Time
9.8 Conclusions
References
Part III Machine Learning for Failure Modeling
10 Extreme Statistics in Memories
10.1 Cell Failure Probability: An Extreme Statistic
10.1.1 Units of Failure Probability
10.1.2 An Example of Extreme Statistics in Memories
10.1.3 Incorporating Redundancy
10.2 Extremes: Tails and maxima
10.2.1 Sample Maximum: Limiting Distributions
10.2.2 Distribution Tail: Limiting Distributions
10.3 Analysis of Tails and Extreme Values
10.3.1 Order Statistics and Quantiles
10.3.1.1 Mean Excess Plot
10.4 Estimating the Tail: Learning the GPD Parameters from Data
10.4.1 Maximum Likelihood Estimation
10.4.2 Probability-Weighted Moment Matching
10.5 Statistical Blockade: Sampling Rare Events
10.5.1 Conditionals and Disjoint Tail Regions
10.5.2 Extremely Rare Events and Their Statistics
10.5.3 A Recursive Formulation of Statistical Blockade
10.6 Conclusions
References
11 Fast Statistical Analysis Using Machine Learning
11.1 Introduction: Logistic Regression-Based Importance Sampling Methodology for Statistical Analysis of Memory Design
11.2 Monte Carlo, Importance Sampling, and Variance Reduction Methods
11.2.1 Monte Carlo
11.2.1.1 Estimating the Fail Probability of a Circuit Design
11.2.2 Variance Reduction
11.2.3 Importance Sampling
11.3 Logistic Regression
11.3.1 Simple Linear Regression Review
11.3.2 Logistic Regression Overview
11.3.2.1 Logistic Regression: Numerical Optimization Techniques
11.4 Proposed Methodology
11.4.1 Methodology Overview
11.4.1.1 Terminology and Definitions
11.4.1.2 Methodology Flow Diagram
11.4.1.3 Regularization Framework
11.4.1.4 Cross-validation
11.5 Application to State-of-the-Art FinFET SRAM Design
11.5.1 Selective Boosting and Write-Assist Circuitry
11.5.2 Experimental Setup
11.5.3 Analysis and Results
11.5.3.1 Phase 1 Model Building: Uniform Sampling Stage
11.5.3.2 Phase 2 Model Prediction: Importance Sampling Stage
11.5.3.3 Yield Estimation and Convergence Analysis
11.6 Conclusions
References
12 Fast Statistical Analysis of Rare Circuit Failure Events
12.1 Introduction
12.2 Subset Simulation
12.3 Scaled-Sigma Sampling
12.4 Conclusions
References
13 Learning from Limited Data in VLSI CAD
13.1 Introduction
13.2 Iterative Feature Search
13.2.1 The Need for Domain Knowledge
13.2.2 The Model Evaluation Step
13.2.3 The Tool Requirement
13.3 Assumptions in Machine Learning
13.4 Traditional Machine Learning
13.4.1 Occam's Razor
13.4.2 Avoid Overfitting
13.5 An Adjusted Machine Learning View
13.5.1 Search for a Hypothesis Space Assumption
13.6 A SAT-Based Implementation
13.6.1 Boolean Learning
13.6.2 Monomial Learning
13.6.3 Hypothesis Space Pruning
13.6.4 SAT Encoding
13.6.5 Effect of the Uniqueness Requirement
13.7 Incorporating Domain Knowledge
13.8 Conclusions
References
Part IV Machine Learning for Analog Design
14 Large-Scale Circuit Performance Modeling by BayesianModel Fusion
14.1 Introduction
14.2 Pre-silicon Validation
14.2.1 Moment Estimation
14.2.2 Distribution Estimation
14.3 Post-silicon Tuning
14.4 Conclusions
References
15 Sparse Relevance Kernel Machine-Based Performance Dependency Analysis of Analog and Mixed-Signal Circuits
15.1 Introduction
15.2 Feature Kernel Weighting
15.2.1 Kernel Methods
15.2.2 Weighting via Atomized Kernel
15.2.3 Learning Model with Feature Kernels
15.3 Sparse Relevance Kernel Machine
15.3.1 Relevance Vector Machine
15.3.2 Bayesian Learning Model for SRKM
15.3.3 Efficient Training Algorithm
15.4 Experiments
15.4.1 Variability Analysis of an LDO
15.4.2 PLL BIST Scheme Optimization
15.4.3 Binary Classification for Functional Check
15.5 Conclusions
References
16 SiLVR: Projection Pursuit for Response Surface Modeling
16.1 Motivation
16.2 Prevailing Response Surface Models
16.2.1 Linear Model
16.2.2 Quadratic Model
16.2.3 PROjection Based Extraction (PROBE)
16.3 Latent Variables and Ridge Functions
16.3.1 Latent Variable Regression
16.3.2 Ridge Functions and Projection Pursuit Regression
16.4 Approximation using ridge functions
16.4.1 Density: What Can Ridge Functions Approximate?
16.4.2 Degree of approximation: how good are they?
16.5 Projection Pursuit Regression
16.5.1 Smoothing and the Bias-Variance Trade-off
16.5.2 Convergence of Projection Pursuit Regression
16.6 SiLVR
16.6.1 The Model
16.6.1.1 Model Complexity
16.6.2 On the Convergence of SiLVR
16.6.3 Interpreting the SiLVR Model
16.6.3.1 Relative Global Sensitivity
16.6.3.2 Input-Referred Correlation
16.6.4 Training SiLVR
16.6.4.1 Initialization Using Spearman's Rank Correlation
16.6.4.2 The Levenberg–Marquardt Algorithm
16.6.4.3 Bayesian Regularization
16.6.4.4 Modified Fivefold Cross-Validation
16.7 Experimental Results
16.7.1 Master–Slave Flip-Flop with Scan Chain
16.7.2 Two-Stage RC-Compensated Opamp
16.7.3 Sub-1V CMOS Bandgap Voltage Reference
16.7.3.1 Training Time
16.8 Conclusions
References
17 Machine Learning-Based System Optimization and Uncertainty Quantification for Integrated Systems
17.1 Introduction
17.2 Optimization Oriented Design Flow
17.3 Black-Box Optimization
17.3.1 Why Machine Learning for Optimization?
17.3.2 Bayesian Optimization Based on Gaussian Process
17.4 Two-Stage Bayesian Optimization
17.4.1 Fast Exploration and Pure Exploitation Stages
17.4.2 Hierarchical Partitioning Scheme
17.4.3 Learning Acquisition Functions
17.4.4 Experiments on Challenge Functions
17.5 Co-optimization of Embedded Inductor and Integrated Voltage Regulator
17.5.1 Buck Converter Efficiency Model
17.5.2 Embedded Inductor Characterization
17.5.3 Optimization Setup
17.5.4 Results
17.6 Uncertainty Quantification
17.6.1 Introduction
17.6.2 Polynomial Chaos Expansion
17.6.2.1 Introduction
17.6.2.2 From Standard to Sparse PC Expansion
17.6.2.3 Post-Processing
17.7 Uncertainty Quantification of the IVR Efficiency
17.7.1 Surrogate Model of the Model Response
17.7.2 Sensitivity Analysis of the Model Response
17.8 Conclusions
References
Part V Machine Learning for System Design and Optimization
18 SynTunSys: A Synthesis Parameter Autotuning System for Optimizing High-Performance Processors
18.1 Introduction
18.2 SynTunSys Architecture
18.2.1 Human Design Time vs. Compute Time
18.2.2 Scenario Composition and Initial Design Space Reduction
18.2.3 SynTunSys Components
18.2.4 Decision Engine Algorithms
18.3 The SynTunSys Decision Engine
18.3.1 The Base Decision Algorithm
18.3.2 The Learning Algorithm
18.4 SynTunSys Results
18.4.1 z13 Case Study Results
18.4.2 14nm Data Mining Results
18.4.3 Decision Algorithm Comparisons
18.5 SynTunSys Enhancements and Future Work
18.5.1 Opportunities for History-Based Enhancements
18.5.1.1 The Jump-Start Algorithm
18.5.1.2 The Recommender System
18.5.2 Improving Compute Footprint Efficiency
18.5.3 A Scheduling System for Multi-Macro Tuning
18.5.3.1 STSS Priority Ranking
18.5.3.2 Second Pass SynTunSys Runs
18.5.3.3 STSS Results
18.6 Related Work
18.7 Conclusions
References
19 Multicore Power and Thermal Proxies Using Least-Angle Regression
19.1 Introduction
19.2 Preliminaries
19.3 Data Collection Platform
19.4 Power Proxies
19.4.1 The Single-Core Case
19.4.2 The Multicore Case
19.5 Temperature Proxies
19.5.1 The Single-Core Case
19.5.2 The Multicore Case
19.6 Proxies Incorporating Sleep States
19.7 Workload Signature
19.8 Core Scaling and Thread Assignment
19.8.1 The Relation of PCs with Power Consumption
19.8.2 Energy-Aware Workload Execution
19.8.3 Autoscaling for Separate Workloads
19.8.4 Autoscaling for Multiple Workloads
19.8.4.1 Scaling Down
19.8.4.2 Scaling Up
19.9 Conclusions
References
20 A Comparative Study of Assertion Mining Algorithmsin GoldMine
20.1 Introduction
20.1.1 Related Work
20.2 Summary of Comparison of Assertion Generation Algorithms in GoldMine
20.3 The GoldMine Principle: Statistics Meet Static
20.4 Background on GoldMine
20.4.1 GoldMine Assertions
20.4.2 Static Analyzer
20.4.3 Data Generator
20.4.4 A-Miner
20.4.5 Formal Verifier
20.4.6 Assertion Evaluator
20.5 Decision Tree-Based Learning
20.6 Best-Gain Decision Forest Algorithm
20.6.1 Algorithm
20.6.2 Example
20.6.3 Analysis
20.7 Coverage Guided Mining Algorithm
20.7.1 Overview of the Algorithm
20.8 PRISM Algorithm
20.9 Experimental Results
20.9.1 Number of Generated Assertions
20.9.2 Sequential Depth of Generated Assertions
20.9.3 Input Space Coverage and Hit Rate of Generated Assertions
20.9.4 Runtime and Memory Usage
20.9.5 RTL Statement Coverage of Generated Assertions
20.9.6 Expectedness Analysis of Generated Assertions
20.9.7 Complexity Analysis of Generated Assertions
20.9.8 Importance Analysis of Generated Assertions
20.9.9 Qualitative Analysis of Sample Assertions
20.9.9.1 Qualitative Comparison of Assertions Generated by Different Algorithms
20.9.9.2 Qualitative Comparison of Automatically Generated Assertions with Manually Written Assertions
20.9.10 Scalability of GoldMine
20.10 Conclusions
References
21 Energy-Efficient Design of Advanced Machine LearningHardware
21.1 Artificial Intelligence and Machine Learning
21.1.1 Neural Networks
21.1.2 Resource Requirements of State-of-the-Art Neural Networks
21.2 Software and Co-design Optimizations
21.2.1 Pruning
21.2.2 Weight Sharing
21.2.3 Compact Network Architectures
21.2.4 Hardware–Software Co-design
21.3 Hardware-Level Techniques
21.3.1 Dataflows and Architectures for Accelerators
21.3.2 Hardware Friendly Strategies for Deep CNN Accelerators
21.3.3 Memory-Efficient Architectures
21.3.4 Hardware Architectural Techniques for Leveraging Sparsity in Neural Networks
21.4 Error Resilience Analysis: DNN-Specific Approximations for Low-Power Accelerators
21.5 Energy-Efficient Hardware Accelerator Design Methodology for Neural Networks
21.6 Efficient Machine Learning Architectures: Challenges and the Way Forward
21.6.1 Optimizing Memory vs. Computations
21.6.2 Neuromorphic Computing
21.6.3 Accuracy vs. Energy Trade-off
21.6.4 Adaptability, (Re-)configurability, and Scalability
21.6.5 Run-Time Evolutionary Algorithms for Designing and Optimizing DNN Architectures
21.6.6 Correct Benchmarking with Fairness and High Fidelity
21.6.7 Open-Source Contributions
References
Index