第1页 / 共405页
第2页 / 共405页
第3页 / 共405页
第4页 / 共405页
第5页 / 共405页
第6页 / 共405页
第7页 / 共405页
第8页 / 共405页
Introduction to Python for Econometrics Statistics and Data Anal....pdf
Introduction
Background
Conventions
Important Components of the Python Scientific Stack
Setup
Using Python
Exercises
Frequently Encountered Problems
register_python.py
Advanced Setup
Python 2.7 vs. 3 (and the rest)
Python 2.7 vs. 3
Intel Math Kernel Library and AMD Core Math Library
Other Variants
Relevant Differences between Python 2.7 and 3
Built-in Data Types
Variable Names
Core Native Data Types
Python and Memory Management
Exercises
Arrays and Matrices
Array
Matrix
1-dimensional Arrays
2-dimensional Arrays
Multidimensional Arrays
Concatenation
Accessing Elements of an Array
Slicing and Memory Management
import and Modules
Calling Functions
Exercises
Basic Math
Operators
Broadcasting
Array and Matrix Addition (+) and Subtraction (-)
Array Multiplication (*)
Matrix Multiplication (*)
Array and Matrix Division (/)
Array Exponentiation (**)
Matrix Exponentiation (**)
Parentheses
Transpose
Operator Precedence
Exercises
Basic Functions and Numerical Indexing
Generating Arrays and Matrices
Rounding
Mathematics
Complex Values
Set Functions
Sorting and Extreme Values
Nan Functions
Functions and Methods/Properties
Exercises
Special Arrays
Exercises
Array and Matrix Functions
Views
Shape Information and Transformation
Linear Algebra Functions
Exercises
Importing and Exporting Data
Importing Data using pandas
Importing Data without pandas
Saving or Exporting Data using pandas
Saving or Exporting Data without pandas
Exercises
Inf, NaN and Numeric Limits
inf and NaN
Floating point precision
Exercises
Logical Operators and Find
>, >=, <, <=, ==, !=
and, or, not and xor
Multiple tests
is*
Exercises
Advanced Selection and Assignment
Numerical Indexing
Logical Indexing
Performance Considerations and Memory Management
Assignment with Broadcasting
Exercises
Flow Control, Loops and Exception Handling
Whitespace and Flow Control
if … elif … else
for
while
try … except
List Comprehensions
Tuple, Dictionary and Set Comprehensions
Exercises
Dates and Times
Creating Dates and Times
Dates Mathematics
Numpy datetime64
Graphics
seaborn
2D Plotting
Advanced 2D Plotting
3D Plotting
General Plotting Functions
Exporting Plots
Exercises
Structured Arrays
Mixed Arrays with Column Names
Record Arrays
pandas
Data Structures
Statistical Function
Time-series Data
Importing and Exporting Data
Graphics
Examples
Custom Function and Modules
Functions
Variable Scope
Example: Least Squares with Newey-West Covariance
Anonymous Functions
Modules
Packages
PYTHONPATH
Python Coding Conventions
Exercises
Listing of econometrics.py
Probability and Statistics Functions
Simulating Random Variables
Simulation and Random Number Generation
Statistics Functions
Continuous Random Variables
Select Statistics Functions
Select Statistical Tests
Exercises
Non-linear Function Optimization
Unconstrained Optimization
Derivative-free Optimization
Constrained Optimization
Scalar Function Minimization
Nonlinear Least Squares
Exercises
String Manipulation
String Building
String Functions
Formatting Numbers
Regular Expressions
Safe Conversion of Strings
File System Operations
Changing the Working Directory
Creating and Deleting Directories
Listing the Contents of a Directory
Copying, Moving and Deleting Files
Executing Other Programs
Creating and Opening Archives
Reading and Writing Files
Exercises
Performance and Code Optimization
Getting Started
Timing Code
Vectorize to Avoid Unnecessary Loops
Alter the loop dimensions
Utilize Broadcasting
Use In-place Assignment
Avoid Allocating Memory
Inline Frequent Function Calls
Consider Data Locality in Arrays
Profile Long Running Functions
Numba
Cython
External Code
Exercises
Executing Code in Parallel
map and related functions
multiprocessing
joblib
IPython's Parallel Cluster
Converting a Serial Program to Parallel
Other Concerns when executing in Parallel
Object Oriented Programming (OOP)
Introduction
Class basics
Building a class for Autoregressions
Exercises
Other Interesting Python Packages
statsmodels
pytz and babel
rpy2
PyTables and h5py
Examples
Estimating the Parameters of a GARCH Model
Estimating the Risk Premia using Fama-MacBeth Regressions
Estimating the Risk Premia using GMM
Outputting LaTeX
Quick Reference
Built-ins
NumPy (numpy)
SciPy
Matplotlib
Pandas
IPython
Introduction to Python for Econometrics, Statistics and Data Analysis
Kevin Sheppard
University of Oxford
Tuesday 5th August, 2014
-
©2012, 2013, 2014 Kevin Sheppard
2
Changes since the Second Edition
Version 2.2.1 (August 2014)
• Fixed typos reported by a reader – thanks to Ilya Sorvachev
Version 2.2 (July 2014)
• Code verified against Anaconda 2.0.1.
• Added diagnostic tools and a simple method to use external code in the Cython section.
• Updated the Numba section to reflect recent changes.
• Fixed some typos in the chapter on Performance and Optimization.
• Added examples of joblib and IPython’s cluster to the chapter on running code in parallel
Version 2.1 (February 2014)
• New chapter introducing object oriented programming as a method to provide structure and orga-
nization to related code.
• Added seaborn to the recommended package list, and have included it be default in the graphics
chapter.
• Based on experience teaching Python to economics students, the recommended installation has
been simplified by removing the suggestion to use virtual environment. The discussion of virtual
environments as been moved to the appendix.
• Rewrote parts of the pandas chapter.
• Code verified against Anaconda 1.9.1.
Version 2.02 (November 2013)
• Changed the Anaconda install to use both create and install, which shows how to install additional
packages.
• Fixed some missing packages in the direct install.
• Changed the configuration of IPython to reflect best practices.
• Added subsection covering IPython profiles.
i
Version 2.01 (October 2013)
• Updated Anaconda to 1.8 and added some additional packages to the installation for Spyder.
• Small section about Spyder as a good starting IDE.
ii
Notes to the 2nd Edition
This edition includes the following changes from the first edition (March 2012):
• The preferred installation method is now Continuum Analytics’ Anaconda. Anaconda is a complete
scientific stack and is available for all major platforms.
• New chapter on pandas. pandas provides a simple but powerful tool to manage data and perform
basic analysis. It also greatly simplifies importing and exporting data.
• New chapter on advanced selection of elements from an array.
• Numba provides just-in-time compilation for numeric Python code which often produces large per-
formance gains when pure NumPy solutions are not available (e.g. looping code).
• Dictionary, set and tuple comprehensions
• Numerous typos
• All code has been verified working against Anaconda 1.7.0.
iii
iv
Contents
1
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Introduction
1.1 Background .
1.2 Conventions .
1.3
1.4 Setup .
.
.
1.5 Using Python .
.
1.6 Exercises
.
.
1.A Frequently Encountered Problems .
1.B register_python.py .
.
1.C Advanced Setup .
.
.
.
.
Important Components of the Python Scientific Stack .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
2 Python 2.7 vs. 3 (and the rest)
.
.
.
.
.
2.1 Python 2.7 vs. 3 .
2.2
2.3 Other Variants .
.
2.A Relevant Differences between Python 2.7 and 3 .
.
Intel Math Kernel Library and AMD Core Math Library .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
3 Built-in Data Types
.
.
.
3.1 Variable Names .
.
.
3.2 Core Native Data Types .
.
3.3 Python and Memory Management .
3.4 Exercises
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
4 Arrays and Matrices
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1-dimensional Arrays
2-dimensional Arrays
4.1 Array .
.
.
.
4.2 Matrix .
.
.
4.3
.
.
4.4
.
.
4.5 Multidimensional Arrays .
.
4.6 Concatenation .
.
.
4.7 Accessing Elements of an Array .
4.8 Slicing and Memory Management
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
v
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1
1
.
2
.
3
.
4
.
6
.
. 17
. 17
. 18
. 19
27
. 27
. 27
. 28
. 29
31
. 31
. 32
. 42
. 44
47
. 47
. 49
. 50
. 51
. 51
. 51
. 52
. 57
4.9
4.10 Calling Functions
4.11 Exercises
.
import and Modules .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
5 Basic Math
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
5.1 Operators .
.
.
.
5.2 Broadcasting .
.
.
5.3 Array and Matrix Addition (+) and Subtraction (-)
5.4 Array Multiplication (*) .
.
.
.
5.5 Matrix Multiplication (*)
.
.
.
5.6 Array and Matrix Division (/) .
.
.
5.7 Array Exponentiation (**) .
.
.
.
5.8 Matrix Exponentiation (**)
.
.
.
5.9 Parentheses .
.
.
.
.
.
5.10 Transpose .
.
.
.
.
.
.
5.11 Operator Precedence .
.
.
.
.
5.12 Exercises
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
6 Basic Functions and Numerical Indexing
.
.
.
.
.
.
.
.
.
6.1 Generating Arrays and Matrices .
6.2 Rounding .
.
.
.
.
6.3 Mathematics .
.
.
.
6.4 Complex Values .
.
.
6.5 Set Functions .
.
.
.
6.6 Sorting and Extreme Values .
.
6.7 Nan Functions .
.
.
6.8 Functions and Methods/Properties
6.9 Exercises
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
7 Special Arrays
7.1 Exercises
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
8 Array and Matrix Functions
.
.
.
.
.
.
.
.
.
.
.
8.1 Views
.
8.2 Shape Information and Transformation
8.3
.
8.4 Exercises
.
Linear Algebra Functions
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
9
Importing and Exporting Data
9.1
.
9.2
.
9.3 Saving or Exporting Data using pandas .
Importing Data using pandas
Importing Data without pandas
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
vi
. 59
. 59
. 61
63
. 63
. 64
. 65
. 66
. 66
. 66
. 66
. 67
. 67
. 67
. 67
. 68
71
. 71
. 74
. 75
. 77
. 77
. 78
. 80
. 81
. 82
83
. 84
85
. 85
. 86
. 93
. 96
99
. 99
. 100
. 106
相关推荐
2023年江西萍乡中考道德与法治真题及答案.doc
2012年重庆南川中考生物真题及答案.doc
2013年江西师范大学地理学综合及文艺理论基础考研真题.doc
2020年四川甘孜小升初语文真题及答案I卷.doc
2020年注册岩土工程师专业基础考试真题及答案.doc
2023-2024学年福建省厦门市九年级上学期数学月考试题及答案.doc
2021-2022学年辽宁省沈阳市大东区九年级上学期语文期末试题及答案.doc
2022-2023学年北京东城区初三第一学期物理期末试卷及答案.doc
2018上半年江西教师资格初中地理学科知识与教学能力真题及答案.doc
2012年河北国家公务员申论考试真题及答案-省级.doc
2020-2021学年江苏省扬州市江都区邵樊片九年级上学期数学第一次质量检测试题及答案.doc
2022下半年黑龙江教师资格证中学综合素质真题及答案.doc
