logo资料库

Bayesian Methods for Hackers Probabilistic Programming and Bayes....pdf

第1页 / 共252页
第2页 / 共252页
第3页 / 共252页
第4页 / 共252页
第5页 / 共252页
第6页 / 共252页
第7页 / 共252页
第8页 / 共252页
资料共252页,剩余部分请下载后查看
Cover
Copyright
Contents
Foreword
Preface
About the Author
1 The Philosophy of Bayesian Inference
1.1 Introduction
1.1.1 The Bayesian State of Mind
1.1.2 Bayesian Inference in Practice
1.1.3 Are Frequentist Methods Incorrect?
1.1.4 A Note on "Big Data"
1.2 Our Bayesian Framework
1.2.1 Example: Mandatory Coin-Flip
1.2.2 Example: Librarian or Farmer?
1.3 Probability Distributions
1.3.1 Discrete Case
1.3.2 Continuous Case
1.3.3 But What Is?
1.4 Using Computers to Perform Bayesian Inference for Us
1.4.1 Example: Inferring Behavior from Text-Message Data
1.4.2 Introducing Our First Hammer: PyMC
1.4.3 Interpretation
1.4.4 What Good Are Samples from the Posterior, Anyway?
1.5 Conclusion
1.6 Appendix
1.6.1 Determining Statistically if the Two s Are Indeed Different?
1.6.2 Extending to Two Switchpoints
1.7 Exercises
1.7.1 Answers
1.8 References
2 A Little More on PyMC
2.1 Introduction
2.1.1 Parent and Child Relationships
2.1.2 PyMC Variables
2.1.3 Including Observations in the Model
2.1.4 Finally. . .
2.2 Modeling Approaches
2.2.1 Same Story, Different Ending
2.2.2 Example: Bayesian A/B Testing
2.2.3 A Simple Case
2.2.4 A and B Together
2.2.5 Example: An Algorithm for Human Deceit
2.2.6 The Binomial Distribution
2.2.7 Example: Cheating Among Students
2.2.8 Alternative PyMC Model
2.2.9 More PyMC Tricks
2.2.10 Example: Challenger Space Shuttle Disaster
2.2.11 The Normal Distribution
2.2.12 What Happened the Day of the Challenger Disaster?
2.3 Is Our Model Appropriate?
2.3.1 Separation Plots
2.4 Conclusion
2.5 Appendix
2.6 Exercises
2.6.1 Answers
2.7 References
3 Opening the Black Box of MCMC
3.1 The Bayesian Landscape
3.1.1 Exploring the Landscape Using MCMC
3.1.2 Algorithms to Perform MCMC
3.1.3 Other Approximation Solutions to the Posterior
3.1.4 Example: Unsupervised Clustering Using a Mixture Model
3.1.5 Don't Mix Posterior Samples
3.1.6 Using MAP to Improve Convergence
3.2 Diagnosing Convergence
3.2.1 Autocorrelation
3.2.2 Thinning
3.2.3 pymc.Matplot.plot()
3.3 Useful Tips for MCMC
3.3.1 Intelligent Starting Values
3.3.2 Priors
3.3.3 The Folk Theorem of Statistical Computing
3.4 Conclusion
3.5 Reference
4 The Greatest Theorem Never Told
4.1 Introduction
4.2 The Law of Large Numbers
4.2.1 Intuition
4.2.2 Example: Convergence of Poisson Random Variables
4.2.3 How Do We Compute Var(Z)?
4.2.4 Expected Values and Probabilities
4.2.5 What Does All This Have to Do with Bayesian Statistics?
4.3 The Disorder of Small Numbers
4.3.1 Example: Aggregated Geographic Data
4.3.2 Example: Kaggle's U.S. Census Return Rate Challenge
4.3.3 Example: How to Sort Reddit Comments
4.3.4 Sorting!
4.3.5 But This Is Too Slow for Real-Time!
4.3.6 Extension to Starred Rating Systems
4.4 Conclusion
4.5 Appendix
4.5.1 Derivation of Sorting Comments Formula
4.6 Exercises
4.6.1 Answers
4.7 References
5 Would You Rather Lose an Arm or a Leg?
5.1 Introduction
5.2 Loss Functions
5.2.1 Loss Functions in the Real World
5.2.2 Example: Optimizing for the Showcase on The Price Is Right
5.3 Machine Learning via Bayesian Methods
5.3.1 Example: Financial Prediction
5.3.2 Example: Kaggle Contest on Observing Dark Worlds
5.3.3 The Data
5.3.4 Priors
5.3.5 Training and PyMC Implementation
5.4 Conclusion
5.5 References
6 Getting Our Priorities Straight
6.1 Introduction
6.2 Subjective versus Objective Priors
6.2.1 Objective Priors
6.2.2 Subjective Priors
6.2.3 Decisions, Decisions . . .
6.2.4 Empirical Bayes
6.3 Useful Priors to Know About
6.3.1 The Gamma Distribution
6.3.2 The Wishart Distribution
6.3.3 The Beta Distribution
6.4 Example: Bayesian Multi-Armed Bandits
6.4.1 Applications
6.4.2 A Proposed Solution
6.4.3 A Measure of Good
6.4.4 Extending the Algorithm
6.5 Eliciting Prior Distributions from Domain Experts
6.5.1 Trial Roulette Method
6.5.2 Example: Stock Returns
6.5.3 Pro Tips for the Wishart Distribution 184
6.6 Conjugate Priors
6.7 Jeffreys Priors
6.8 Effect of the Prior as N Increases
6.9 Conclusion
6.10 Appendix
6.10.1 Bayesian Perspective of Penalized Linear Regressions
6.10.2 Picking a Degenerate Prior
6.11 References
7 Bayesian A/B Testing
7.1 Introduction
7.2 Conversion Testing Recap
7.3 Adding a Linear Loss Function
7.3.1 Expected Revenue Analysis
7.3.2 Extending to an A/B Experiment
7.4 Going Beyond Conversions: t-test
7.4.1 The Setup of the t-test
7.5 Estimating the Increase
7.5.1 Creating Point Estimates
7.6 Conclusion
7.7 References
Glossary
A
B
C
D
E
F
G
L
M
O
P
S
T
W
Index
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
R
S
T
U
V
W
Z
ptg21761562
Bayesian Methods for Hackers
ptg21761562T he Addison-Wesley Data and Analytics Series provides readers with practical knowledge for solving problems and answering questions with data. Titles in this series primarily focus on three areas:1.Infrastructure: how to store, move, and manage data2.Algorithms: how to mine intelligence or make predictions based on data3. Visualizations: how to represent data and insights in a meaningful and compelling wayThe series aims to tie all three of these areas together to help the reader build end-to-end systems for fighting spam; making recommendations; building personalization; detecting trends, patterns, or problems; and gaining insight from the data exhaust of systems and user interactions.Visit informit.com/awdataseries for a complete list of available publications.Make sure to connect with us!informit.com/socialconnectThe Addison-Wesley Data and Analytics Series
Bayesian Methods for Hackers Probabilistic Programming and Bayesian Inference Cameron Davidson-Pilon New York • Boston • Indianapolis • San Francisco Toronto • Montreal • London • Munich • Paris • Madrid Capetown • Sydney • Tokyo • Singapore • Mexico City ptg21761562
Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and the publisher was aware of a trademark claim, the designations have been printed with initial capital letters or in all capitals. The author and publisher have taken care in the preparation of this book, but make no expressed or implied warranty of any kind and assume no responsibility for errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of the use of the information or programs contained herein. For information about buying this title in bulk quantities, or for special sales opportunities (which may include electronic versions; custom cover designs; and content particular to your business, training goals, marketing focus, or branding interests), please contact our corporate sales department at corpsales@pearsoned.com or (800) 382-3419. For government sales inquiries, please contact governmentsales@pearsoned.com. For questions about sales outside the United States, please contact international@pearsoned.com. Visit us on the Web: informit.com/aw Library of Congress Cataloging-in-Publication Data Davidson-Pilon, Cameron. Bayesian methods for hackers : probabilistic programming and bayesian inference / Cameron Davidson-Pilon. pages cm Includes bibliographical references and index. ISBN 978-0-13-390283-9 (pbk.: alk. paper) 1. Penetration testing (Computer security)–Mathematics. 2. Bayesian statistical decision theory. 3. Soft computing. I. Title. QA76.9.A25D376 2015 006.3–dc23 2015017249 Copyright © 2016 Cameron Davidson-Pilon All rights reserved. Printed in the United States of America. This publication is protected by copyright, and permission must be obtained from the publisher prior to any prohibited reproduction, storage in a retrieval system, or transmission in any form or by any means, electronic, mechanical, photocopying, recording, or likewise. To obtain permission to use material from this work, please submit a written request to Pearson Education, Inc., Permissions Department, 200 Old Tappan Road, Old Tappan, New Jersey 07675, or you may fax your request to (201) 236-3290. The code throughout and Chapters 1 through 6 in this book is released under the MIT License. ISBN-13: 978-0-13-390283-9 ISBN-10: 0-13-390283-8 Text printed in the United States on recycled paper at RR Donnelley in Crawfordsville, Indiana. First printing, October 2015 ptg21761562
O This book is dedicated to many important relationships: my parents, my brothers, and my closest friends. Second to them, it is devoted to the open-source community, whose work we consume every day without knowing. O ptg21761562
This page intentionally left blank ptg21761562
Contents Foreword xiii Preface xv Acknowledgments xvii About the Author xix 1 The Philosophy of Bayesian Inference 1 1.1 The Bayesian State of Mind 1 Introduction 1 1.1.1 1.1.2 Bayesian Inference in Practice 3 1.1.3 Are Frequentist Methods Incorrect? 4 1.1.4 A Note on “Big Data” 4 1.2 Our Bayesian Framework 5 1.2.1 Example: Mandatory Coin-Flip 5 1.2.2 Example: Librarian or Farmer? 6 Probability Distributions 8 1.3.1 Discrete Case 9 1.3.2 Continuous Case 10 1.3.3 But What Is λ? 12 Using Computers to Perform Bayesian Inference for Us 12 1.4.1 Example: Inferring Behavior from Text-Message Data 12 1.4.2 Introducing Our First Hammer: PyMC 14 1.4.3 Interpretation 18 1.4.4 What Good Are Samples from the Posterior, Anyway? 18 Conclusion 20 Appendix 20 1.6.1 Determining Statistically if the Two λs Are Indeed Different? 20 1.6.2 Extending to Two Switchpoints 22 Exercises 24 1.7.1 References 25 Answers 24 1.3 1.4 1.5 1.6 1.7 1.8 ptg21761562
分享到:
收藏