u
d
e
.
o
g
e
w
s
o
.
s
c
.
e
e
g
/
/
:
p
t
t
h
Scalable IO in Java
Doug Lea
State University of New York at Oswego
dl@cs.oswego.edu
http://gee.cs.oswego.edu
Outline
" Scalable network services
" Event-driven processing
" Reactor pattern
Basic version
Multithreaded versions
Other variants
u
d
e
.
o
g
e
w
s
o
.
s
c
.
e
e
g
/
/
:
p
t
t
h
" Walkthrough of java.nio nonblocking IO APIs
Network Services
" Web services, Distributed Objects, etc
" Most have same basic structure:
Read request
Decode request
Process service
Encode reply
Send reply
" But differ in nature and cost of each step
XML parsing, File transfer, Web page
generation, computational services, ...
u
d
e
.
o
g
e
w
s
o
.
s
c
.
e
e
g
/
/
:
p
t
t
h
Classic Service Designs
u
d
e
.
o
g
e
w
s
o
.
s
c
.
e
e
g
/
/
:
p
t
t
h
client
client
Server
client
read
read
read
decode compute encode
handler
decode compute encode
handler
decode compute encode
handler
send
send
send
Each handler may be started in its own thread
Classic ServerSocket Loop
class Server implements Runnable {
public void run() {
try {
ServerSocket ss = new ServerSocket(PORT);
while (!Thread.interrupted())
new Thread(new Handler(ss.accept())).start();
// or, single-threaded, or a thread pool
} catch (IOException ex) { /* ... */ }
}
static class Handler implements Runnable {
final Socket socket;
Handler(Socket s) { socket = s; }
public void run() {
try {
byte[] input = new byte[MAX_INPUT];
socket.getInputStream().read(input);
byte[] output = process(input);
socket.getOutputStream().write(output);
} catch (IOException ex) { /* ... */ }
}
private byte[] process(byte[] cmd) { /* ... */ }
}
}
Note: most exception handling elided from code examples
u
d
e
.
o
g
e
w
s
o
.
s
c
.
e
e
g
/
/
:
p
t
t
h
Scalability Goals
" Graceful degradation under increasing load
(more clients)
" Continuous improvement with increasing
resources (CPU, memory, disk, bandwidth)
" Also meet availability and performance goals
Short latencies
Meeting peak demand
Tunable quality of service
" Divide-and-conquer is usually the best
approach for achieving any scalability goal
u
d
e
.
o
g
e
w
s
o
.
s
c
.
e
e
g
/
/
:
p
t
t
h
Divide and Conquer
" Divide processing into small tasks
Each task performs an action without blocking
" Execute each task when it is enabled
Here, an IO event usually serves as trigger
decode compute encode
send
read
handler
" Basic mechanisms supported in java.nio
Non-blocking reads and writes
Dispatch tasks associated with sensed IO events
" Endless variation possible
A family of event-driven designs
u
d
e
.
o
g
e
w
s
o
.
s
c
.
e
e
g
/
/
:
p
t
t
h
Event-driven Designs
" Usually more efficient than alternatives
Fewer resources
" Don't usually need a thread per client
Less overhead
" Less context switching, often less locking
But dispatching can be slower
" Must manually bind actions to events
" Usually harder to program
Must break up into simple non-blocking actions
" Similar to GUI event-driven actions
" Cannot eliminate all blocking: GC, page faults, etc
Must keep track of logical state of service
u
d
e
.
o
g
e
w
s
o
.
s
c
.
e
e
g
/
/
:
p
t
t
h