The Python/C API
Release 2.7.12
Guido van Rossum
and the Python development team
September 09, 2016
Python Software Foundation
Email: docs@python.org
CONTENTS
1 Introduction
.
.
.
.
.
.
.
Include Files .
1.1
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2 Objects, Types and Reference Counts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.3
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.4
1.5 Debugging Builds
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.
Exceptions .
Embedding Python .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
2 The Very High Level Layer
3 Reference Counting
4 Exception Handling
4.1 Unicode Exception Objects
.
4.2
.
4.3
4.4
.
Recursion Control
.
Standard Exceptions .
String Exceptions .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
5 Utilities
.
.
.
.
.
.
.
System Functions .
Process Control .
.
.
Importing Modules .
5.1 Operating System Utilities .
.
5.2
.
.
5.3
.
5.4
.
.
5.5 Data marshalling support .
.
5.6
5.7
5.8
5.9
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Parsing arguments and building values
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
String conversion and formatting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Reflection .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Codec registry and support functions
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
6 Abstract Objects Layer
6.1 Object Protocol .
.
6.2 Number Protocol .
6.3
Sequence Protocol
6.4 Mapping Protocol
.
6.5
6.6 Old Buffer Protocol
Iterator Protocol
.
.
.
.
.
.
.
.
.
.
.
7 Concrete Objects Layer
7.1
7.2 Numeric Objects .
7.3
Sequence Objects .
7.4 Mapping Objects .
7.5 Other Objects .
.
Fundamental Objects .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.
.
.
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3
3
4
7
9
9
11
15
17
21
22
22
23
25
25
25
26
26
29
30
36
38
39
41
41
45
48
50
51
51
53
53
54
61
86
88
8 Initialization, Finalization, and Threads
105
Initializing and finalizing the interpreter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
Process-wide parameters .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
8.1
8.2
.
.
.
i
8.3
8.4
.
8.5 Asynchronous Notifications .
8.6
.
8.7 Advanced Debugger Support .
Thread State and the Global Interpreter Lock . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
Sub-interpreter support .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
Profiling and Tracing .
.
.
.
.
.
.
.
.
.
9 Memory Management
9.1 Overview .
.
9.2 Memory Interface
9.3
.
Examples .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
117
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
10 Object Implementation Support
121
10.1 Allocating Objects on the Heap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
10.2 Common Object Structures
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
.
10.3 Type Objects .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
.
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
10.4 Number Object Structures .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
10.5 Mapping Object Structures .
10.6 Sequence Object Structures
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
10.7 Buffer Object Structures .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
10.8 Supporting Cyclic Garbage Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
A Glossary
145
B About these documents
153
B.1 Contributors to the Python Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
C History and License
155
C.1 History of the software .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
C.2 Terms and conditions for accessing or otherwise using Python . . . . . . . . . . . . . . . . . . . 155
C.3 Licenses and Acknowledgements for Incorporated Software . . . . . . . . . . . . . . . . . . . . 158
.
.
.
.
.
D Copyright
Index
171
173
ii
The Python/C API, Release 2.7.12
This manual documents the API used by C and C++ programmers who want to write extension modules or embed
Python. It is a companion to extending-index, which describes the general principles of extension writing but does
not document the API functions in detail.
CONTENTS
1
The Python/C API, Release 2.7.12
2
CONTENTS
CHAPTER
ONE
INTRODUCTION
The Application Programmer’s Interface to Python gives C and C++ programmers access to the Python interpreter
at a variety of levels. The API is equally usable from C++, but for brevity it is generally referred to as the
Python/C API. There are two fundamentally different reasons for using the Python/C API. The first reason is to
write extension modules for specific purposes; these are C modules that extend the Python interpreter. This is
probably the most common use. The second reason is to use Python as a component in a larger application; this
technique is generally referred to as embedding Python in an application.
Writing an extension module is a relatively well-understood process, where a “cookbook” approach works well.
There are several tools that automate the process to some extent. While people have embedded Python in other
applications since its early existence, the process of embedding Python is less straightforward than writing an
extension.
Many API functions are useful independent of whether you’re embedding or extending Python; moreover, most
applications that embed Python will need to provide a custom extension as well, so it’s probably a good idea to
become familiar with writing an extension before attempting to embed Python in a real application.
1.1 Include Files
All function, type and macro definitions needed to use the Python/C API are included in your code by the following
line:
#include "Python.h"
This implies inclusion of the following standard headers:
, , ,
, and (if available).
Note: Since Python may define some pre-processor definitions which affect the standard headers on some sys-
tems, you must include Python.h before any standard headers are included.
All user visible names defined by Python.h (except those defined by the included standard headers) have one of
the prefixes Py or _Py. Names beginning with _Py are for internal use by the Python implementation and should
not be used by extension writers. Structure member names do not have a reserved prefix.
Important: user code should never define names that begin with Py or _Py. This confuses the reader, and jeop-
ardizes the portability of the user code to future Python versions, which may define additional names beginning
with one of these prefixes.
The header files are typically installed with Python.
these are located in the directo-
ries prefix/include/pythonversion/ and exec_prefix/include/pythonversion/, where
prefix and exec_prefix are defined by the corresponding parameters to Python’s configure script and
version is sys.version[:3]. On Windows, the headers are installed in prefix/include, where prefix
is the installation directory specified to the installer.
To include the headers, place both directories (if different) on your compiler’s search path for includes. Do
not place the parent directories on the search path and then use #include ; this
will break on multi-platform builds since the platform independent headers under prefix include the platform
specific headers from exec_prefix.
On Unix,
3
The Python/C API, Release 2.7.12
C++ users should note that though the API is defined entirely using C, the header files do properly declare the
entry points to be extern "C", so there is no need to do anything special to use the API from C++.
1.2 Objects, Types and Reference Counts
Most Python/C API functions have one or more arguments as well as a return value of type PyObject*. This
type is a pointer to an opaque data type representing an arbitrary Python object. Since all Python object types
are treated the same way by the Python language in most situations (e.g., assignments, scope rules, and argument
passing), it is only fitting that they should be represented by a single C type. Almost all Python objects live on
the heap: you never declare an automatic or static variable of type PyObject, only pointer variables of type
PyObject* can be declared. The sole exception are the type objects; since these must never be deallocated, they
are typically static PyTypeObject objects.
All Python objects (even Python integers) have a type and a reference count. An object’s type determines what
kind of object it is (e.g., an integer, a list, or a user-defined function; there are many more as explained in types).
For each of the well-known types there is a macro to check whether an object is of that type; for instance,
PyList_Check(a) is true if (and only if) the object pointed to by a is a Python list.
1.2.1 Reference Counts
The reference count is important because today’s computers have a finite (and often severely limited) memory size;
it counts how many different places there are that have a reference to an object. Such a place could be another
object, or a global (or static) C variable, or a local variable in some C function. When an object’s reference
count becomes zero, the object is deallocated. If it contains references to other objects, their reference count
is decremented. Those other objects may be deallocated in turn, if this decrement makes their reference count
become zero, and so on. (There’s an obvious problem with objects that reference each other here; for now, the
solution is “don’t do that.”)
Reference counts are always manipulated explicitly. The normal way is to use the macro Py_INCREF() to
increment an object’s reference count by one, and Py_DECREF() to decrement it by one. The Py_DECREF()
macro is considerably more complex than the incref one, since it must check whether the reference count becomes
zero and then cause the object’s deallocator to be called. The deallocator is a function pointer contained in
the object’s type structure. The type-specific deallocator takes care of decrementing the reference counts for
other objects contained in the object if this is a compound object type, such as a list, as well as performing any
additional finalization that’s needed. There’s no chance that the reference count can overflow; at least as many
bits are used to hold the reference count as there are distinct memory locations in virtual memory (assuming
sizeof(Py_ssize_t) >= sizeof(void*)). Thus, the reference count increment is a simple operation.
It is not necessary to increment an object’s reference count for every local variable that contains a pointer to an
object. In theory, the object’s reference count goes up by one when the variable is made to point to it and it goes
down by one when the variable goes out of scope. However, these two cancel each other out, so at the end the
reference count hasn’t changed. The only real reason to use the reference count is to prevent the object from being
deallocated as long as our variable is pointing to it. If we know that there is at least one other reference to the
object that lives at least as long as our variable, there is no need to increment the reference count temporarily.
An important situation where this arises is in objects that are passed as arguments to C functions in an extension
module that are called from Python; the call mechanism guarantees to hold a reference to every argument for the
duration of the call.
However, a common pitfall is to extract an object from a list and hold on to it for a while without incrementing
its reference count. Some other operation might conceivably remove the object from the list, decrementing its
reference count and possible deallocating it. The real danger is that innocent-looking operations may invoke
arbitrary Python code which could do this; there is a code path which allows control to flow back to the user from
a Py_DECREF(), so almost any operation is potentially dangerous.
A safe approach is to always use the generic operations (functions whose name begins with PyObject_,
PyNumber_, PySequence_ or PyMapping_). These operations always increment the reference count of
the object they return. This leaves the caller with the responsibility to call Py_DECREF() when they are done
with the result; this soon becomes second nature.
4
Chapter 1.
Introduction