Titl
®
Tsi721™ Windows Device Driver
and API User Reference Guide
®
Formal Status
October 30, 2013
About this Document
This document targets software developers who are familiar with the RapidIO specification, and therefore, does not provide
introductory information on RapidIO terminology and operations. If required, the latest RapidIO specification can be obtained
from www.rapidio.org.
Topics discussed include the following:
• Tsi721 Device Driver Package
• Tsi721 Device Driver
• Application Programming Interface
• Code Examples
Revision History
October 30, 2013, Formal
Minor update to Tsi721 Device Driver Package.
May 6, 2013, Formal
Updated to clarify open /close API parameters and to include new test code sample.
March 25, 2013, Formal
This is the first document release.
Tsi721 Device Driver Package
The Tsi721 Windows Device Driver Package consists of following components:
1. Kernel-Mode Device Driver binaries (tsi721.sys) for Windows 7 (32- and 64-bit) and Windows XP
2. User mode API DLL (tsi721_api.dll) which simplifies access and use of the device driver
3. The code samples that demonstrate use of the APIs from this package:
•
•
The first code sample targets the Tsi721 Evaluation Board but can be easily adopted for the user’s own configuration. A
pre-built binary tested with the Tsi721 Evaluation Board is also included.
The second code sample can be used with two Tsi721 devices directly connected using an S-RIO link.
4. Readme file that provides information about updates and last minute documentation changes
5. Tsi721 Windows Device Driver and API (this document)
The most recent version of the Tsi721 Device Driver Package is v.1.03.
Tsi721 Windows Device Driver and API
Formal Status
Integrated Device Technology, Inc.
2
October 30, 2013
This document is confidential and is subject to an NDA.
Tsi721 Device Driver
Driver Overview
The Tsi721 device driver for Windows is designed using Microsoft Windows Kernel-Mode Driver Framework.
Due to specifics of the Tsi721 hardware and the general RapidIO interconnect architecture, the device driver was designed to
provide device-specific IOCTL functions for all its operations including data transfers to/from RapidIO endpoints: traditional
read and write interfaces are not suitable for RapidIO fabric topology.
The Tsi721 device driver supports overlapping I/O requests and multiple parallel request queues. This provides the capability
to combine multiple asynchronous event notifications with regular synchronous data transfer and register access operations.
Through its device-specific IOCTL codes, the device driver supports the following operations:
• Read/write access to local Tsi721 registers located in PCI Express Configuration space
• Read/write access to local Tsi721 registers located in PCI Express I/O memory space
• Maintenance read/write operations from/to remote S-RIO device’s CSRs (using Tsi721 BDMA channel)
• Addressed data read/write operations from/to remote endpoints connected to RapidIO fabric (NREAD, NWRITE, SWRITE
using Tsi721 BDMA channel)
• Sending and receiving doorbell notifications to/from RapidIO endpoints. Inbound doorbell queue size is configured to store
up to 512 entries.
• Sending and receiving RapidIO mailbox messages (MBOX0–MBOX3). Inbound messaging supports only queues for
messages with matching destination IDs.
• Mapping of inbound RapidIO memory address in to the local system memory buffer
• Receiving Port-Write notifications from attached RapidIO fabric components (switches and endpoints)
Tsi721 features that are not supported:
• PCI Express to S-RIO memory mapping windows. This Tsi721 hardware feature is not supported due to challenging error
handling for failed requests that return response (NREAD, MAINT_READ). This requires implementation of a custom
Machine Check handler. For more information, see Tsi721 User Manual.
Inbound messaging queues for messages with non-matching destination IDs
•
Differences from Linux Kernel RapidIO Support
Users who are familiar with Linux kernel RapidIO Subsystem implementation will not see a level of service similar to one that is
available for RapidIO in Linux kernel. Because Windows OS does not offer built-in support for RapidIO, users have to
implement their own S-RIO fabric management functionality. None of these upper layer services (such as enumeration and
discovery, error management initialization, and routing setup) are included into this package. The Tsi721 device driver simply
provides interfaces sufficient to implement these services for Windows (in user-mode). The Tsi721 device driver for Windows
also differs from RapidIO “master port” by implementing interfaces to user mode applications instead of kernel-mode interfaces
in Linux.
Tsi721 Windows Device Driver and API
Formal Status
Integrated Device Technology, Inc.
3
October 30, 2013
This document is confidential and is subject to an NDA.
Device-Specific IOCTL Codes
To communicate with devices and perform RapidIO input/output operations, the Tsi721 device driver provides a set of
device-specific IOCTL codes that can be passed to the device driver along with required parameters by calling the
general-purpose user-mode interface function DeviceIoControl.
If more information is required, see the following Microsoft articles:
• Device Input and Output Control (IOCTL) – http://msdn.microsoft.com/EN-US/library/windows/desktop/
aa363219(v=vs.85).aspx
• Calling DeviceIoControl – http://msdn.microsoft.com/en-us/library/windows/desktop/aa363147(v=vs.85).aspx
Assuming an understanding of basic IOCTL operations by the end user, the following reference sections describe input and
output parameters as they are exchanged between the device driver and a caller of DeviceIoControl function:
•
Input parameters are shown as an array of DWORDs in[n], which is referenced as input buffer pointer of DeviceIoControl
function
• Output parameters are shown as an array of DWORDs out[n], which is referenced as output buffer pointer of
DeviceIoControl function
• Size of input and output buffers in use has to be specified as it is required by format of DeviceIoControl function
Because the Tsi721 device driver supports overlapped requests, the DeviceIoControl function must provide a pointer to a valid
OVERLAPPED data structure as it is described in documentation: http://msdn.microsoft.com/EN-US/library/windows/
desktop/aa363216(v=vs.85).aspx.
The following section provides examples of using the Tsi721 device driver’s IOCTL codes.
To simplify the interface to the driver and reduce the need in understanding device-specific IOCTL codes, this device driver
package provides a user mode API that is described later in this document. Users may consider using DeviceIoControl
function and listed device-specific IOCTL codes to develop their own versions of API or extend the included one.
All IOCTL codes listed below are defined in file tsi721ioctl.h that is included into this device driver package.
IOCTLs for Basic Register Access
IOCTL_TSI_REG_READ
Reads from the Tsi721 memory-mapped register (or block of registers) using 32-bit PCIe memory read operation.
Parameters:
•
•
The number of registers to read is defined by the size of the IOCTL request output buffer (always a multiple of four bytes).
in[0] – Register offset in TSI721 PCIe memory-mapped register space
out[0..n] – Values read from register (or block of registers)
IOCTL_TSI_REG_WRITE
Writes into TSI721 memory-mapped register using 32-bit PCIe memory write operation.
Parameters:
•
•
The output buffer pointer for this IOCTL request should be NULL and size = 0.
in[0] – Register offset in TSI721 PCIe memory-mapped register space
in[1] – 32-bit value to write into the register
Tsi721 Windows Device Driver and API
Formal Status
Integrated Device Technology, Inc.
4
October 30, 2013
This document is confidential and is subject to an NDA.
IOCTL_TSI_MNT_READ
Reads form a control register of remote RapidIO device using a RapidIO MAINTENANCE read request (single 32-bit
operation).
Parameters:
•
•
•
•
in[0] – Destination ID of a target RapidIO device
in[1] – Hop count to the target RapidIO device (0xff for endpoints)
in[2] – Register offset in the CSR space
out[0] – 32-bit value read from the target RapidIO device
IOCTL_TSI_MNT_WRITE
Writes into a control register of remote RapidIO device using RapidIO MAINTENANCE write request
Parameters:
•
•
•
•
The output buffer pointer for this IOCTL request should be NULL and size = 0.
in[0] – Destination ID of a target RapidIO device
in[1] – Hop count to the target RapidIO device (0xff for endpoints)
in[2] – Register offset
in[3] – 32-bit value to write
IOCTL_TSI_PCFG_READ
Reads form a Tsi721 register in PCI Express Configuration space.
Parameters:
•
•
in[0] – Register offset in TSI721 PCI Express Configuration space
out[0] – 32-bit value read from TSI721 register in PCI Express Configuration space
IOCTL_TSI_PCFG_WRITE
Writes into a Tsi721 register in PCI Express Configuration space.
Parameters:
•
•
The output buffer pointer for this IOCTL request should be NULL and size = 0.
in[0] – Register offset in TSI721 PCI Express Configuration space
in[1] – 32-bit value to write into register
Data Transfer IOCTLs
IOCTL_TSI_SRIO_WRITE
Writes a data block into a target RapidIO device memory space using RapidIO NWRITE/SWRITE request(s). This IOCTL
request uses the Tsi721 BDMA engine to transfer data directly from a user mode buffer specified as an output buffer for the
DeviceIoControl function.
Parameters:
•
•
•
in[0] – Destination ID of a target RapidIO device
in[1] – Bits 63:32 of 66-bit starting address in the target RapidIO device
in[2] – Bits 31:00 of 66-bit starting address in the target RapidIO device
Tsi721 Windows Device Driver and API
Formal Status
Integrated Device Technology, Inc.
5
October 30, 2013
This document is confidential and is subject to an NDA.
•
•
in[3] – DMA request-specific control word (see tsi721ioctl.h for a definition of the DMA_REQ_CTRL structure)
out[…] – User mode source data buffer
IOCTL_TSI_SRIO_READ
Reads a data block from a target RapidIO device memory space using RapidIO NREAD request(s). This IOCTL request uses
the Tsi721 BDMA engine to transfer read data directly into a user-mode buffer specified as an output buffer for DeviceIoControl
function.
Parameters:
•
•
•
•
•
in[0] – Destination ID of a target RapidIO device
in[1] – Bits 63:32 of 66-bit starting address in the target RapidIO device
in[2] – Bits 31:00 of 66-bit starting address in the target RapidIO device
in[3] – DMA request-specific control word (see tsi721ioctl.h for definition of DMA_REQ_CTRL structure)
out[…] – User mode destination data buffer
IOCTL_TSI_IBW_BUFFER_GET
Returns data from the driver's internal kernel-mode buffer allocated for data exchange using inbound RapidIO
NREAD/NWRITE requests initiated by an external device (using Tsi721 PCIe bus mastering capability, see
IOCTL_TSI_R2P_WIN_SET).
This IOCTL request places data directly into a user-mode buffer specified as an output buffer for DeviceIoControl function.
Parameters:
•
•
•
in[0] – Inbound mapping (SR2PC) window number (0–7) associated with buffer
in[1] – Starting offset of data block within a data buffer
out[…] – User-mode destination data buffer
IOCTL_TSI_IBW_BUFFER_PUT
Places a data block into the driver's internal kernel-mode buffer allocated for data exchange using inbound RapidIO
NREAD/NWRITE requests initiated by an external device (using Tsi721 PCIe bus mastering capability, see
IOCTL_TSI_R2P_WIN_SET).
This IOCTL sends data directly from a user-mode buffer specified as an output buffer for DeviceIoControl function.
Parameters:
•
•
•
in[0] – Inbound mapping (SR2PC) window number (0–7) associated with buffer
in[1] – Starting offset of data block within a data buffer
out[…] – User-mode source data buffer
Doorbell Operations
IOCTL_TSI_DB_SEND
Sends a doorbell message to a specified target RapidIO device
Parameters:
•
•
• No output data – Output buffer pointer for this IOCTL request should be NULL and size = 0
in[0] – Destination ID of a target RapidIO device
in[1] – 16-bit S-RIO doorbell INFO field (lower 16 bits) combined with CRF bit (bit 31)
Tsi721 Windows Device Driver and API
Formal Status
Integrated Device Technology, Inc.
6
October 30, 2013
This document is confidential and is subject to an NDA.
IOCTL_TSI_DB_WAIT
Requests inbound doorbell event notification. If at the moment of processing of this IOCTL request by the driver inbound
doorbells already present in Tsi721 hardware doorbell queue, request will be completed returning available doorbell messages
(up to size of an output buffer provided by caller).
If inbound doorbell message is not available, the device driver places the IOCTL request into the device driver’s wait queue for
doorbell notifications and returns ERROR_IO_PENDING to a caller. The caller should wait for the IOCTL request completion
on the event object provided as part of OVERLAPPED data structure parameter to the DeviceIoControl function. For more
information on asynchronous notification wait techniques, see the corresponding API functions.
Parameters:
•
•
in[0] – Inbound doorbell queue number (must be 0 at this moment)
out[0..n] – Doorbell messages placed into the output buffer. Number of returned doorbell messages is defined by size of
returned data (in bytes) divided by size of doorbell element (8 bytes).
Return value ERROR_IO_PENDING indicates that given IOCTL request was placed into wait notification queue and will be
released when a new inbound doorbell is available or when the IOCTL request is cancelled.
The Tsi721 Evaluation Board test code sample demonstrates how to implement a notification thread that waits on an event
object associated with this IOCTL request.
IOCTL_TSI_DB_CHECK
Checks Tsi721 inbound doorbell queue for received inbound doorbell notification(s). Compared to the IOCTL_TSI_DB_WAIT
request, this is a synchronous check that immediately returns even if there is no inbound doorbell message in the queue. This
IOCTL request can be used if a user prefers polling for doorbell notifications instead of handling asynchronous events.
Parameters:
•
•
in[0] – Inbound doorbell queue number (must be 0 at this moment)
out[0] – Number of doorbell messages in the queue (0 if no doorbells available)
IOCTL_TSI_DB_GET
Fetches available inbound doorbell messages. This IOCTL request is intended to be used together with
IOCTL_TSI_DB_CHECK when a user implements an inbound doorbell polling routine.
Parameters:
•
•
in[0] – Inbound doorbell queue number (must be 0 at this moment)
out[0..n] – Doorbell messages placed into output buffer
Messaging Operations
IOCTL_TSI_MSG_SEND
Sends RapidIO mailbox message to the specified target RapidIO device.
This IOCTL request relies on data transfer directly from a user-mode buffer specified as an output buffer for DeviceIoControl
function. Because Tsi721 outbound messaging DMA channels do not support scatter/gather (SG) operations, message source
buffer in user space must be within a single page of physical memory. Otherwise, if message buffer crosses page boundary
and has more than one SG element the device driver will report an error. To ensure that a message buffer is within a single
memory page, page-aligned allocation has to be used (4KB alignment boundary).
For example, _aligned_malloc function can be used as shown in code line:
msgBufPtr = (PUCHAR)_aligned_malloc(MSG_SIZE, PAGE_SIZE);
Tsi721 Windows Device Driver and API
Formal Status
Integrated Device Technology, Inc.
7
October 30, 2013
This document is confidential and is subject to an NDA.
Parameters:
•
•
•
in[0] – RapidIO messaging mailbox number (0..3)
in[1] – Destination ID of a target RapidIO device
out[…] – User-mode source message buffer
IOCTL_TSI_IB_MSG_DEV_ID_SET
Sets Inbound Messaging destination ID for Tsi721 controller. For information about inbound messaging queues handling, see
“Inbound Messaging Engine deviceID Register” in the Tsi721 User Manual.
The Tsi721 device driver in this package supports only queues for messages with matching
destination IDs.
Parameters:
•
in[0] – Inbound messaging device destination ID for a local RapidIO controller (tsi721) (valid device ID: 16-bit value 0–0xffff,
see “Inbound Messaging Engine deviceID Register” in the Tsi721 User Manual). Normally this destination ID is set to match
the Base Destination ID assigned to the specific Tsi721.
• No output data – Output buffer pointer for this IOCTL request should be NULL and size = 0.
IOCTL_TSI_IB_MSG_DEV_ID_GET
Returns an inbound messaging device destination ID assigned to tsi721.
Parameters:
• No input data – Input buffer pointer for this IOCTL request should be NULL and size = 0
•
out[0] – Device destination ID assigned to the specified local Tsi721 controller
IOCTL_TSI_MSG_ADDRXBUF
Adds a user-mode message receive buffer into an inbound message queue for the specified MBOX.
This request is placed into a waiting queue until a new inbound RapidIO message arrives. A physical address of the output
buffer associated with request is added into the Tsi721 list of free buffers available for incoming messages. After inbound buffer
is successfully registered, this IOCTL request returns with status ERROR_IO_PENDING. A calling application may register
multiple receive buffers while the number of buffers per MBOX does not exceed inbound message queue size (currently 128
messages per MBOX). If a specified inbound MBOX queue is full, DeviceIoControl function will return status ERROR_BUSY.
When one of registered message buffers receives a message, the device driver completes the corresponding request and
notifies a caller using the provided OVERLAPPED structure.
A caller should wait for the request completion on the event object provided as part of OVERLAPPED data structure parameter
to the DeviceIoControl function, or using I/O completion port mechanism provided by Windows for asynchronous I/O. For more
information on asynchronous notification wait techniques, see the corresponding API functions.
To avoid buffer starvation of inbound messaging channels it is recommended to register maximum possible number of inbound
buffers for each MBOX to have them available when message arrives.
A caller should not expect that inbound messages for a specific MBOX will be returned in the order of buffer registration.
This IOCTL request relies on data placement directly into user-mode buffer specified as an output buffer for the
DeviceIoControl function. Because Tsi721 inbound messaging DMA channels do not support scatter/gather (SG) operations,
destination message buffer in user space must be within a single page of physical memory. Otherwise, if message buffer
crosses page boundary and has more than one SG element the device driver will report an error. To ensure that a message
buffer is within a single memory page, page-aligned allocation has to be used (4KB alignment boundary).
Tsi721 Windows Device Driver and API
Formal Status
Integrated Device Technology, Inc.
8
October 30, 2013
This document is confidential and is subject to an NDA.