第1页 / 共473页
第2页 / 共473页
第3页 / 共473页
第4页 / 共473页
第5页 / 共473页
第6页 / 共473页
第7页 / 共473页
第8页 / 共473页
OS X and iOS Kernel Programming 无水印pdf.pdf
Cover
Contents at a Glance
Contents
About the Authors
About the Technical Reviewers
Acknowledgments
Introduction
Operating System Fundamentals
The Role of the Operating System
Process Management
Process Address Spaces
Operating System Services
Virtual Memory
Scheduling
Hardware and Drivers
Summary
Mac OS X and iOS
Programming APIs
Supported Platforms
64-bit Operating System
iOS
The XNU Kernel
Kernel Extensions (KEXTs)
Mach
Tasks and Threads
Scheduling
Mach IPC: Ports and Messages
Mach Exceptions
Time Management
Memory Management
Task Address Space
VM Maps and Entries
The Physical Map
VM Objects
Examining a Task's Address Space
Pagers
Memory Allocation in Mach
The BSD Layer
System Calls
Networking
File Systems
The Virtual File System
Unified Buffer Cache
The I/O Kit
The Libkern Library
The Platform Expert
Xcode and the Kernel Development Environment
Language of Choice: C++
Xcode
"Hello World" Kernel Extension
Loading and Unloading Kernel Extensions
Using Console to View Output
Summary
The I/O Kit Framework
The I/O Kit Model
Object Relationship
The Info.plist File
The Driver Class
IORegistryExplorer
The Kernel Library: libkern
OSObject
Container Classes
Summary
Interacting with Drivers from Applications
The I/O Kit Framework
Finding a Driver
Observing Device Removal
Modifying Driver Properties
State-Based Interaction
Summary
Memory Management
Types of Memory
CPU Physical Address
Bus Physical Addresses
User and Kernel Virtual Addresses
Memory Ordering: Big vs. Little Endian
32-bit vs. 64-bit Memory Addressing
Memory Allocation
Low-Level Allocation Mechanisms
The Mach Zone Allocator
The kalloc Family
Memory Allocation in BSD
I/O Kit Memory Allocation
Allocating Memory with the C++ New Operator
Memory Descriptors
The IOBufferMemoryDescriptor
Other Memory Descriptors
Mapping Memory
Mapping Memory from a User Space Task into Kernel Space
The IOMemoryMap Class
Mapping Memory from the Kernel to a User Space Task
Mapping Memory to a Specific User Space Task
Physical Address Mapping
Summary
Synchronization and Threading
Synchronization Primitives
Atomic Operations
Locking
Spin locks
Mutexes
Condition Variables
Read/Write Mutexes
Synchronizing Asynchronous Events: Work Loops
IOCommandGate
Timers
Releasing Work Loops
Kernel Threads
Summary
Universal Serial Bus
USB Architecture
USB Transfer Speeds
Host Controllers
The USB Protocol
Endpoints
USB Descriptors
USB Device Classes
I/O Kit USB Support
USB Device and Driver Handling
Loading USB Drivers
USB Prober
Driver Example: USB Mass Storage Device Driver
Driver Startup
Handling Device Removals
Enumerating Interfaces
Enumerating Endpoints
Performing Device Requests
Control Requests
Performing I/O to Bulk and Interrupt Endpoints
Dealing with Errors and Pipe Stalls
Isochronous I/O
Asynchronous Requests
Summary
PCI Express and Thunderbolt
PCI Express
Thunderbolt
ExpressCard
Configuration Space Registers
PCI in I/O Kit
Matching and Loading Drivers
Driver Example: A Simple PCI Driver
Accessing Configuration Space Registers
Accessing the Extended Configuration Space
Searching for Capabilities Registers
PCI I/O Memory Regions
Enumerating I/O Regions
Mapping and Accessing Device Memory Regions
Accessing I/O Space
Handling Device Removal
Interrupts
I/O Kit Interrupt Mechanisms
Registering to Receive Interrupts
Enabling Message Signaled Interrupts
Handling Primary Interrupts
Handling Secondary Interrupts
Direct Memory Access
Translating Physical Addresses to Bus Addresses
Preparing Memory for DMA
Building a Scatter/Gather List
The IODMACommand Class
Summary
Power Management
The I/O Registry Power Plane
Power Management in the I/O Kit
Responding to Power State Changes
Responding to Power State Changes
Requesting Power State Changes
Handling Device Idle
Observing Device Power State Changes
Putting It All Together
Summary
Serial Port Drivers
Mac OS X Serial Port Architecture Overview
Serial Port Drivers
Manually Instantiating a Driver Object
Implementing the IOSerialDriverSync Class
Serial Port State
Serial Port Events
Serial Data Transfer
Accessing a Serial Port from User Space
Summary
Audio Drivers
An Introduction to Digital Audio and Audio Devices
Core Audio
I/O Kit Audio Support
Implementing an Audio Driver
Driver and Hardware Initialization
Registering Audio Controls
Implementing an Audio Engine
I/O Engine Initialization
Creating and Initializing Audio Streams
Handling Format Changes
Clipping and Converting Samples
Starting and Stopping the Audio Engine
Engine Operation: Handling Interrupts and Timestamps
Additional Audio Engine Functionality
Summary
Networking
Network Memory Buffers
Working with Memory Buffers
Network Kernel Extensions
Kernel Control KPI
Socket Filters
Building an Application-Level Firewall Using Socket Filters
AppWall Operation and Data Structures
Attaching and Detaching the Filter
Handling Connections
Socket Data Input and Output
Internet Protocol Filters
Interface Filters
Debugging and Testing Network Extensions
Networking in the I/O Kit
Building a Simple Ethernet Controller Driver
The Design of MyEthernetDriver
Driver Initialization and Startup
Medium and Status Selection
Configuring the Device Hardware Address
Enabling and Disabling the Device
Transmitting Network Packets
Receiving Packets
Taking MyEthernetDriver for a Test-Drive
Summary
Storage Systems
Transport Layer Drivers
The IOBlockStorageDevice Interface
Building a RAM Disk Device
Partition Schemes
Implementing a Sample Partition Scheme
The Media Content Hint Property
Media Filter Drivers
A Sample Filter Scheme for Encryption
Creating a Custom GUID Partition Table
Summary
User-Space USB Drivers
Behind the Scenes
The IOUSBLib Framework
Handling Asynchronous Operations
The IOUSBDeviceInterface Class
The IOUSBInterfaceInterface Class
Property Methods
Endpoint Data Transfer Methods
Low-Latency Isochronous Transfers
Summary
Debugging
Common Types of Problems
Kernel Panics
Debugging Mechanisms
Recovering from Crashes During Boot
Tracing with IOLog()
Printing Stack Traces
Remote Tracing over FireWire
Remote Kernel Core Dumps
KDB
Remote Debugging with GDB over Ethernet or FireWire
Configuring the Host Machine
Attaching to the Remote Target
Debugging Using FireWire
Live Debugging of a Running Kernel
Debugging Using a Virtual Machine
Debugging in the Kernel Using GDB
Kernel GDB Macros
Creating Symbol Information for KEXTs
Debugging KEXTs with GDB
Understanding Kernel Panic Logs
x86-64 Calling Conventions
Diagnosing Hung Processes with Activity Monitor
Finding Memory and Resource Leaks
Advanced Kernel Programming
SSE and Floating Point in the Kernel
Multi-Function Drivers
Writing I/O Kit Families
Kernel Control KPI
Kernel Control Registration
Client Connections
Getting and Setting Options
Accessing Kernel Controls from User Space
Working with Processes in the Kernel
Loading Resources
Beyond KEXT Resources
Notifications from Kernel Drivers
427Summary
Deployment
Installing and Loading Kernel Extensions
Loading Preferences and Settings
Versioning Kernel Extensions
Testing and Quality Assurance
Packaging KEXTs and Software
Building a Package for the Hello World Kernel Extension
Adding Contents to the Package
Configuring the Package
Building the Package
Uninstalling Packages
Summary
Index
Numbers and Symbols
A
B
C
D
E
F
G
H
I, J
K
L
M
N
O
P, Q
R
S
T
U
V
W
X, Y
Z
BOOKS FOR PROFESSIONALS BY PROFESSIONALS®
OS X and iOS Kernel Programming combines essential operating system and
kernel architecture knowledge with a highly practical approach that will
help you write effective kernel-level code. You’ll learn fundamental concepts
such as memory management and thread synchronization, as well as the I/O
Kit framework. You’ll also learn how to write your own kernel-level extensions,
such as device drivers for USB and Thunderbolt devices, including networking,
storage and audio drivers.
OS X and iOS Kernel Programming provides an incisive and complete introduc-
tion to the XNU kernel, which runs iPhones, iPads, iPods, and Mac OS X servers
and clients. Then, you’ll expand your horizons to examine Mac OS X and iOS
system architecture. Understanding Apple’s operating systems will allow you to
write efficient device drivers, such as those covered in the book, using I/O Kit.
With OS X and iOS Kernel Programming, you’ll:
• Discover classical kernel architecture topics such as memory
management and thread synchronization
• Become well-versed in the intricacies of the kernel development
process by applying kernel debugging and profiling tools
• Learn how to deploy your kernel-level projects and how to successfully
package them
• Write code that interacts with hardware devices
• Examine easy to understand example code that can also be used in your
own projects
• Create network filters
Whether you’re a hobbyist, student, or professional engineer, turn to OS X and
iOS Kernel Programming and find the knowledge you need to start developing
your own device drivers and applications that control hardware devices.
Companion
eBook
Available
O
S
X
a
n
d
i
O
S
K
e
r
n
e
l
P
r
o
g
r
a
m
m
n
g
i
COMPANION eBOOK
SOURCE CODE ONLINE
www.apress.com
Shelve in
Programming / Mac / Mobile
User level:
Intermediate–Advanced
H
a
l
v
o
r
s
e
n
l
C
a
r
k
e
Master kernel programming for
efficiency and performance
OS X and iOS Kernel
Programming
Ole Henry Halvorsen | Douglas Clarke
>
m
o
c
.
k
o
o
b
e
w
o
w
w
w
w
<
k
o
o
B
e
.
!
w
o
W
m
o
r
f
l
d
a
o
n
w
o
D
For your convenience Apress has placed some of the front
matter material after the index. Please use the Bookmarks
and Contents at a Glance links to access them.
iv
Contents at a Glance
About the Authors................................................................................................ xiv
About the Technical Reviewers ............................................................................ xv
Acknowledgments ............................................................................................... xvi
Introduction ........................................................................................................ xvii
Chapter 1: Operating System Fundamentals ..........................................................1
Chapter 2: Mac OS X and iOS................................................................................15
Chapter 3: Xcode and the Kernel Development Environment ..............................39
Chapter 4: The I/O Kit Framework ........................................................................51
Chapter 5: Interacting with Drivers from Applications........................................69
Chapter 6: Memory Management .........................................................................99
Chapter 7: Synchronization and Threading ........................................................119
Chapter 8: Universal Serial Bus ..........................................................................141
Chapter 9: PCI Express and Thunderbolt ............................................................173
Chapter 10: Power Management ........................................................................205
Chapter 11: Serial Port Drivers...........................................................................223
Chapter 12: Audio Drivers...................................................................................249
Chapter 13: Networking......................................................................................275
Chapter 14: Storage Systems .............................................................................319
Chapter 15: User-Space USB Drivers ..................................................................357
Chapter 16: Debugging .......................................................................................381
Chapter 17: Advanced Kernel Programming ......................................................411
Chapter 18: Deployment .....................................................................................429
Index ...................................................................................................................443
Introduction
Kernel development can be a daunting task and is very different from programming traditional user
applications. The kernel environment is more volatile and complex. Extraordinary care must be taken to
ensure that kernel code is free of bugs because any issue may have serious consequences to the stability,
security, and performance of the system. This book covers the fundamentals necessary to begin
programming in the kernel. We cover kernel development from a theoretical and practical point of view.
We cover concepts fundamental to kernel development such as virtual memory and synchronization, as
well as more practical knowledge. The book primarily focuses on Mac OS X, however the XNU kernel is
also used by iOS, and hence the theoretical material in this book will also apply to it. By far the most
common reason for doing development within the kernel’s execution environment is to implement a
device driver for controlling internal or external hardware devices. Because of this, much of the focus of
this book is centred on the development of device drivers. The primary framework for device driver
development in the XNU kernel is I/O Kit, which we cover extensively. As theory becomes boring quickly
we have provided working code samples which you can play with to learn more or use as a starting point
for your own drivers.
We hope you have as much fun reading this book as we have enjoyed writing it.
Who Is This Book For?
The book was written for anyone interested in Apple’s iOS and Mac OS X operating systems, with a focus
on practical kernel development, especially driver devel. Regardless of whether you are a hobbyist,
student, or professional engineer, we hope to provide you with material of interest. While the focus is on
kernel programming and development, we will cover many theoretical aspects of OS technology and
provide a detailed overview of the OS X and iOS kernel environments. The aim of the book is to provide
the knowledge necessary to start developing your own kernel extensions and drivers. We will focus in
particular on the I/O Kit framework for writing device drivers and extensions, but we will also cover
general knowledge that will give you a deeper understanding of how I/O Kit interacts with the OS. If you
are mainly interested in developing OS X or iOS user applications, this book may not be for you. We will
not cover Cocoa or any other framework used for developing end-user applications. This book covers
kernel-programming topics such as driver and kernel extension development on Apple’s OS X and iOS
platform.
Some knowledge of operating system internals will be useful in understanding the concepts
discussed in this book. Having completed an introductory computer science or engineering course will
be a helpful starting point. Additionally, knowledge of at least one programming language will be
required in order to understand examples throughout the book. Since we focus on I/O Kit, which is
written in a subset of C++ called Embedded C++, it would be highly beneficial to have some experience
with C++ (or at least C) to make the most of this book. The book does not cover general programming
topics or theory. We will briefly cover some fundamentals of OS theory to provide a context for further
discussions.
xvii
INTRODUCTION
Book Structure
The following is a brief description of each chapter in this book:
Chapter 1, Operating System Fundamentals. Details the functionality of an operating system and
its role in managing the computer’s hardware resources. We describe the purpose of device drivers and
when they are needed, and introduce the differences between programming in the kernel environment
as compared to standard application development.
Chapter 2, Mac OS X and iOS. Provides a brief overview of the technical structure of XNU, the kernel
used by Mac OS X and iOS.
Chapter 3, Xcode and the Kernel Development Environment. Provides an overview of the
development tools provided by Apple for Mac OS X and iOS development. The chapter ends with a short
“Hello world” kernel extension.
Chapter 4, The I/O Kit Framework. Introduces the I/O Kit framework that provides the driver model
for Mac OS X and its object-oriented architecture. We explain how the I/O Kit finds the appropriate
device driver to manage a hardware device. We demonstrate a generic device driver to illustrate the basic
structure of any I/O Kit driver.
Chapter 5, Interacting with Drivers from Applications. Explains how application code can access a
kernel driver. We demonstrate how to search and match against a specific driver as well as how to install
a notification to wait for the arrival of a driver or a particular device. We will show how an application
can send commands to a driver and watch for events sent by the driver.
Chapter 6, Memory Management. Provides an overview of kernel memory management and the
different types of memory that a driver needs to work with. We describe the differences between physical
and kernel virtual addresses and user-space memory. We also introduce the reader to the concepts such
as memory descriptors and memory mapping.
Chapter 7, Synchronization and Threading. Describes the fundamentals of synchronization and
why it is a necessity for every kernel driver. We discuss the usage of kernel locking mechanisms such as
IOLock and IOCommandGate and their appropriate use. We explain how a typical driver requires
synchronization between its own threads, user-space threads, and hardware interrupts. We discuss the
kernel facilities for creating kernel threads and asynchronous timers.
Chapter 8, USB Drivers. Introduces the reader to the architecture of USB and how a driver
interfaces with them. We provide an overview of the I/O Kit USB API and the classes it provides for
enumerating devices and transferring data to or from a USB device. We also discuss steps needed to
support device removal and provide an example to show how a driver can enumerate resources such as
pipes.
Chapter 9, PCI and Thunderbolt. Provides an overview of the PCI architecture. We also describe the
concepts that are unique to PCI drivers, such as memory-mapped I/O, high-speed data transfer through
Direct Memory Access (DMA), and handling of device interrupts. We give an overview of the IOPCIDevice
class that the I/O Kit provides for accessing and configuring PCI devices. We also discuss the related and
more recent Thunderbolt technology.
Chapter 10, Power Management. Describes the methods that drivers need to implement in order to
allow the system to enter low power states such as machine sleep. We also describe advanced power
management that a driver can implement if it wishes to place its hardware into a low power state after a
period of inactivity.
Chapter 11, Serial Port Drivers. Describes how to implement a serial port driver on Mac OS X. We
introduce relevant data structures such as circular queues and techniques for managing data flow
through blocking I/O and notification events. We show how a user application can enumerate and
access a serial port driver.
xviii
INTRODUCTION
Chapter 12, Audo Drivers. Discusses how system-wide audio input and output devices can be
developed using the IOAudioFamily framework. We demonstrate a simple virtual audio device that
copies audio output to its input.
Chapter 13, Network Drivers. Describes how a network interface can be implemented using the
IONetworkingFamily. We also cover how to write network filters to filter, block, and modify network
packets. The chapter concludes with an example of how to write an Ethernet driver.
Chapter 14, Storage Drivers. Covers the storage driver stack on Mac OS X that provides support for
storage devices such as disks and CDs. We describe the drivers at each layer of the storage stack,
including how to write a RAM disk, a partition scheme, and a filter driver that provides disk encryption.
Chapter 15, User space USB Drivers. Describes how certain drivers can be implemented entirely
inside a user application. We describe the advantages to this approach and also when this may not be
applicable.
Chapter 16, Debugging. Contains practical information on how to debug drivers, as well as
common problems and pitfalls. It will enable a reader to work backwards from a kernel crash report to a
location in their code, a common scenario facing a kernel developer. We will discuss the tools OS X
provides to enable this, such as the GNU debugger (GDB).
Chapter 17, Advanced Kernel Programming. Explores some of the more advanced topics in kernel
programming, such as utilizing SSE and floating point or implementing advanced driver architectures.
Chapter 18, Deployment. Concludes the book by describing how to distribute a driver to the end
user. We cover the use of the Apple installation system for both first-time installation and upgrades. The
chapter includes practical tips on how to avoid common driver installation problems.
xix
C H A P T E R 1
Operating System Fundamentals
The role of an operating system is to provide an environment in which the user is able to run application
software. The applications that users run rely on services provided by the operating system to perform
tasks while they execute, in many cases without the user—or even the programmer—giving much
thought to them. For an application to read a file from disk, for example, the programmer simply needs
to call a function that the operating system provides. The operating system handles the specific steps
required to perform that read. This frees the application programmer from having to worry about the
differences between reading a file that resides on the computer’s internal hard disk or a file on an
external USB flash drive; the operating system takes care of such matters.
Most programmers are familiar with developing code that is run by the user and perhaps uses a
framework such as Cocoa to provide a graphical user interface with which to interact with the user. All of
the applications available on the Mac or iPhone App Store fit into this category. This book is not about
writing application software, but rather about writing kernel extensions—that is, code that provides
services to applications. Two possible situations in which a kernel extension is necessary are allowing
the operating system to work with custom hardware devices and adding support for new file systems.
For example, a kernel extension could allow a new USB audio device to be used by iTunes or allow an
Ethernet card to provide an interface for networking applications, as shown in Figure 1-1. A file system
kernel extension could allow a hard disk formatted on a Windows computer to mount on a Mac as if it
were a standard Mac drive.
1
CHAPTER 1 OPERATING SYSTEM FUNDAMENTALS
Figure 1-1. The network interfaces listed in the Mac OS X system preferences represent network kernel
extensions.
An important role of the operating system is to manage the computer’s hardware resources, such as
memory and the CPU, and peripherals, such as disk storage and the keyboard. The collection of
hardware devices that the operating system needs to support varies greatly from machine to machine.
The hardware configuration of a MacBook Air is very different to that of a Mac Pro, although they both
run the same operating system. To allow the operating system to support multiple hardware
configurations without becoming bloated, the code required to support each hardware component is
packaged into a special type of kernel extension known as a driver. This modularity allows the operating
system to load drivers on demand, depending on the hardware that is present on the system. This
approach also allows for drivers to be installed into the system by vendors to support their custom
hardware. The standard installation of Mac OS X comes with over one hundred drivers, of which only a
subset is needed to run a particular system.
Developing a kernel extension is very different from writing an application. The execution of an
application tends to be driven by events originating from the user. The application runs when the user
launches it; it may then wait for the user to click a button or select a menu item, at which point the
application handles that request. Kernel extensions, on the other hand, have no user interface and do
not interact with the user. They are loaded by the operating system, and are called by the operating
system to perform tasks that it could not perform by itself, such as when the operating system needs to
access a hardware device that the kernel extension is driving.
2
相关推荐
2023年江西萍乡中考道德与法治真题及答案.doc
2012年重庆南川中考生物真题及答案.doc
2013年江西师范大学地理学综合及文艺理论基础考研真题.doc
2020年四川甘孜小升初语文真题及答案I卷.doc
2020年注册岩土工程师专业基础考试真题及答案.doc
2023-2024学年福建省厦门市九年级上学期数学月考试题及答案.doc
2021-2022学年辽宁省沈阳市大东区九年级上学期语文期末试题及答案.doc
2022-2023学年北京东城区初三第一学期物理期末试卷及答案.doc
2018上半年江西教师资格初中地理学科知识与教学能力真题及答案.doc
2012年河北国家公务员申论考试真题及答案-省级.doc
2020-2021学年江苏省扬州市江都区邵樊片九年级上学期数学第一次质量检测试题及答案.doc
2022下半年黑龙江教师资格证中学综合素质真题及答案.doc
