2018/2/22
500 Lines or Less | Introduction
Introduction
Michael DiBernardo
This is the fourth volume in the Architecture of Open Source Applications series, and the first to not feature the words
"open source applications" anywhere in the title.
The first three volumes in the series were about big problems that big programs have to solve. For an engineer who is
early in their career, it may be a challenge to understand and build upon programs that are much bigger than a few
thousand lines of code, so, while big problems can be interesting to read about, they can also be challenging to learn
from.
500 Lines or Less focuses on the design decisions that programmers make in the small when they are building
something new. The programs you will read about in this book were all written from scratch for this purpose (although
several of them were inspired by larger projects that the authors had worked on previously).
Before reading each chapter, we encourage you to first think about how you might solve the problem. What design
considerations or constraints do you think the author is going to consider important? What abstractions do you expect
to see? How do you think the problem is going to be decomposed? Then, when reading the chapter, try to identify
what surprised you. It is our hope that you will learn more by doing this than by simply reading through each chapter
from beginning to end.
Writing a useful program in fewer than 500 lines of source code---without resorting to cheap tricks---is a challenging
exercise in itself; writing one to be read for pedagogical purposes when neatly rendered in a printed book is even
tougher. As such, the editors have occasionally taken liberties with some of the source formatting when porting it into
the book. The original source for each chapter can be found in the code subdirectory of its project folder.
We hope that the experiences of the authors in this book will help you grow out of your comfort zone in your own
programming practice.
---Michael DiBernardo
Contributors
Michael DiBernardo (editorial): Michael DiBernardo is an engineer and director of delivery at Wave, and a past PyCon
Canada chair. He writes at mikedebo.ca.
Amy Brown (editorial): Amy Brown is a freelance editor based in Toronto. She specializes in science and academic
editing, and working with self-publishing authors. She co-edited the Architecture of Open Source Applications books
with Greg Wilson.
Dethe Elza (Blockcode): Dethe is a geek dad, aesthetic programmer, mentor, and creator of the Waterbear visual
programming tool. He co-hosts the Vancouver Maker Education Salons and wants to fill the world with robotic origami
rabbits.
Malini Das (CI): Malini is a software engineer who is passionate about developing quickly (but safely!), and solving
cross-functional problems. She has worked at Mozilla as a tools engineer and is currently honing her skills at Twitch.
Dustin J. Mitchell (Cluster): Dustin is an open source software developer and release engineer at Mozilla. He has
worked on projects as varied as a host configuration system in Puppet, a Flask-based web framework, unit tests for
firewall configurations, and a continuous integration framework in Twisted Python.
Daniel Rocco (Contingent): Daniel loves Python, coffee, craft, stout, object and system design, bourbon, teaching,
trees, and Latin guitar. Thrilled that he gets to write Python for a living, he is always on the lookout for opportunities to
learn from others in the community, and to contribute by sharing knowledge. He is a frequent speaker at PyAtl on
introductory topics, testing, design, and shiny things; he loves seeing the spark of delight in people's eyes when
someone shares a surprising or beautiful idea. Daniel lives in Atlanta with a microbiologist and four aspiring
rocketeers.
file:///C:/Users/Limited/Desktop/limited/500%20line%20or%20less/0.html
1/5
2018/2/22
500 Lines or Less | Introduction
Brandon Rhodes (Contingent): Brandon Rhodes started using Python in the late 1990s, and for 17 years has
maintained the PyEphem library for amateur astronomers. He works at Dropbox, has taught Python programming
courses for corporate clients, consulted on projects like the New England Wildflower Society's "Go Botany" Django
site, and will be the chair of the PyCon conference in 2016 and 2017. Brandon believes that well-written code is a
form of literature, that beautifully formatted code is a work of graphic design, and that correct code is one of the most
transparent forms of thought.
A. Jesse Jiryu Davis (Crawler): Jesse is a staff engineer at MongoDB in New York. He wrote Motor, the async
MongoDB Python driver, and he is the lead developer of the MongoDB C Driver and a member of the PyMongo team.
He contributes to asyncio and Tornado. He writes at emptysqua.re.
Guido van Rossum (Crawler): Guido is the creator of Python, one of the major programming languages on and off the
web. The Python community refers to him as the BDFL (Benevolent Dictator For Life), a title straight from a Monty
Python skit.
Dann Toliver (Dagoba): Dann enjoys building things, like programming languages, databases, distributed systems,
communities of smart friendly humans, and pony castles with his two-year-old.
Taavi Burns (DBDB): As the newest bass (and sometimes tenor) in Countermeasure, Taavi strives to break the
mould... sometimes just by ignoring its existence. This is certainly true through the diversity of workplaces in his
career: IBM (doing C and Perl), FreshBooks (all the things), Points.com (doing Python), and now at PagerDuty (doing
Scala). Aside from that—when not gliding along on his Brompton folding bike—you might find him playing Minecraft
with his son or engaging in parkour (or rock climbing, or other adventures) with his wife. He knits continental.
Leo Zovic: Leo (better known online as inaimathi) is a recovering graphic designer who has professionally written
Scheme, Common Lisp, Erlang, Javascript, Haskell, Clojure, Go, Python, PHP and C. He currently blogs about
programming, plays board games and works at a Ruby-based startup in Toronto, Ontario.
Dr. Christian Muise (Flow shop): Dr. Muise is a Research Fellow with the Model-based Embedded and Robotic
Systems group at MIT's Computer Science and Artificial Intelligence Laboratory. He is interested in a variety of topics
including AI, data-driven projects, mapping, graph theory, and data visualization, as well as Celtic music, carving,
soccer, and coffee.
Yoav Rubin (CircleDB): Yoav is a Senior Software Engineer at Microsoft, and prior to that was a Research Staff
Member and a Master Inventor at IBM Research. He works now in the domain of data security in the cloud, and in the
past his work focused on developing cloud- or web-based development environments. Yoav holds an MSc in Medical
Research in the field of Neuroscience and BSc in Information Systems Engineering.
Cate Huston (Image filters): Cate is a developer and entrepreneur focused on mobile. She’s lived and worked in the
UK, Australia, Canada, China and the United States, as an engineer at Google, an Extreme Blue intern at IBM, and a
ski instructor. Cate speaks internationally on mobile development, and her writing has been published on sites as
varied as Lifehacker, The Daily Beast, The Eloquent Woman and Model View Culture. She co-curates Technically
Speaking, blogs at Accidentally in Code and is @catehstn on Twitter.
Allison Kaptur (Interpreter): Allison is an engineer at Dropbox, where she helps maintain one of the largest networks
of Python clients in the world. Before Dropbox, she was a facilitator at the Recurse Center, a writers' retreat for
programmers in New York. She's spoken at PyCon North America about Python internals, and loves weird bugs.
Erick Dransch (Modeller): Erick is a software developer and 2D and 3D computer graphics enthusiast. He has worked
on video games, 3D special effects software, and computer-aided design tools. If it involves simulating reality,
chances are he'd like to learn more about it. You can find him online at erickdransch.com.
Carl Friedrich Bolz (Object model): Carl is a researcher at King's College London and is broadly interested in the
implementation and optimization of all kinds of dynamic languages. He is one of the core authors of PyPy/RPython
and has worked on implementations of Prolog, Racket, Smalltalk, PHP and Ruby.
Marina Samuel (OCR): Marina is an engineer at Mozilla and a current MSc student in Applied Computing (Artifical
Intelligence) at the University of Toronto. She hopes to one day build robots that will take over the planet.
Dessy Daskalov (Pedometer): Dessy is an engineer by trade, an entrepreneur by passion, and a developer at heart.
She's currently the CTO and co-founder of Nudge Rewards. When she’s not busy building product with her team, she
can be found teaching others to code, attending or hosting a Toronto tech event, and online at dessydaskalov.com
and @dess_e.
file:///C:/Users/Limited/Desktop/limited/500%20line%20or%20less/0.html
2/5
2018/2/22
500 Lines or Less | Introduction
Eunsuk Kang (Same-origin policy): Eunsuk is a PhD candidate and a member of the Software Design Group at MIT.
He received his SM (Master of Science) in Computer Science from MIT (2010), and a Bachelor of Software
Engineering from the University of Waterloo (2007). His research projects have focused on developing tools and
techniques for software modeling and verification, with applications to security and safety-critical systems.
Santiago Perez (Same-origin policy): Santiago is a PhD student in the Software Design Group at MIT. He received his
SM in Computer Science from MIT (2015), and an undergraduate degree from ITBA (2011). He used to work at
Google, developing frameworks and tools to make engineers more productive (2012). He currently spends most of
his time thinking about design and version control.
Daniel Jackson (Same-origin policy): Daniel is a professor in the Department of Electrical Engineering and Computer
Science at MIT, and leads the Software Design Group in the Computer Science and Artificial Intelligence Laboratory.
He received an MA from Oxford University (1984) in Physics, and his SM (1988) and PhD (1992) in Computer
Science from MIT. He was a software engineer for Logica UK Ltd. (1984-1986), Assistant Professor of Computer
Science at Carnegie Mellon University (1992-1997), and has been at MIT since 1997. He has broad interests in
software engineering, especially in development methods, design and specification, formal methods, and safety-
critical systems.
Jessica B. Hamrick (Sampler): Jess is a PhD student at UC Berkeley where she studies human cognition by
combining probabilistic models from machine learning with behavioral experiments from cognitive science. In her
spare time, Jess is a core contributor to IPython and Jupyter. She also holds a BS and MEng in Computer Science
from MIT.
Audrey Tang (Spreadsheet): A self-educated programmer and translator, Audrey works with Apple as an independent
contractor on cloud service localization and natural language technologies. Audrey has previously designed and led
the first working Perl 6 implementation, and served in computer language design committees for Haskell, Perl 5, and
Perl 6. Currently Audrey is a full-time g0v contributor and leads Taiwan’s first e-Rulemaking project.
Leah Hanson (Static analysis): Leah Hanson is a proud alum of Hacker School and loves helping people learn about
Julia. She blogs at blog.leahhanson.us and tweets at @astrieanna.
Ned Batchelder (Template engine): Ned is a software engineer with a long career, currently working at edX to build
open source software to educate the world. He's the maintainer of coverage.py, an organizer of Boston Python, and
has spoken at many PyCons. He blogs at nedbatchelder.com. He once had dinner at the White House.
Greg Wilson (Web server): Greg is the founder of Software Carpentry, a crash course in computing skills for scientists
and engineers. He has worked for 30 years in both industry and academia, and is the author or editor of several
books on computing, including the 2008 Jolt Award winner Beautiful Code and the first two volumes of The
Architecture of Open Source Applications. Greg received a PhD in Computer Science from the University of
Edinburgh in 1993.
Acknowledgments
The Architecture of Open Source Applications series would not exist without the hard work of Amy Brown and Greg
Wilson. This particular book would not have been possible without the incredible efforts of our army of technical
reviewers:
Amber Yust
Andrew Gwozdziewycz
Andrew Kuchling
Andrew Svetlov
Andy Shen
Anton Beloglazov
Ben Trofatter
Borys Pierov
Carise Fernandez
Charles Stanhope
Chris AtLee
Chris Seaton
Cyryl Płotnicki-Chudyk
Dan Langer
Dan Shapiro
file:///C:/Users/Limited/Desktop/limited/500%20line%20or%20less/0.html
3/5
2018/2/22
500 Lines or Less | Introduction
David Pokorny
Eric Bouwers
Frederic De Groef
Graham Lee
Gregory Eric Sanderson
James O'Beirne
Jan de Baat
Jana Beck
Jessica McKellar
Jo Van Eyck
Joel Crocker
Johan Thelin
Johannes Fürmann
John Morrissey
Joseph Kaptur
Josh Crompton
Joshua T. Corbin
Kevin Huang
Maggie Zhou
Marc Towler
Marcin Milewski
Marco Lancini
Mark Reid
Matthias Bussonnier
Max Mautner
Meggin Kearney
Mike Aquino
Natalie Black
Nick Presta
Nikhil Almeida
Nolan Prescott
Paul Martin
Piotr Banaszkiewicz
Preston Holmes
Pulkit Sethi
Rail Aliiev
Ronen Narkis
Rose Ames
Sina Jahan
Stefan Turalski
William Lachance
Chris Seaton, John Morrissey, and Natalie Black deserve extended thanks for going above and beyond in their
technical reviewing. The quantity and depth of their reviews was instrumental in moving the book forward at several
sticking points.
We are very grateful to PagerDuty for their financial support.
Contributing
If you'd like to report errors or translate the content into other languages, please open an issue.
Colophon
The cover font is Museo from the exljibris foundry, by Jos Buivenga.
The front cover photo is composed of twenty-three separate focus-stacked images of watch gear assemblies. The
picture was taken by Kellar Wilson.
This book was built with open source software (with the exception of the cover). Programs like LaTeX, Pandoc, and
Python were especially helpful.
file:///C:/Users/Limited/Desktop/limited/500%20line%20or%20less/0.html
4/5
2018/2/22
500 Lines or Less | Introduction
file:///C:/Users/Limited/Desktop/limited/500%20line%20or%20less/0.html
5/5
2018/2/22
500 Lines or Less | Blockcode: A visual programming toolkit
Blockcode: A visual
programming toolkit
Dethe Elza
Dethe is a geek dad, aesthetic programmer, mentor, and creator of the Waterbear visual programming tool. He co-
hosts the Vancouver Maker Education Salons and wants to fill the world with robotic origami rabbits.
In block-based programming languages, you write programs by dragging and connecting blocks that represent parts
of the program. Block-based languages differ from conventional programming languages, in which you type words
and symbols.
Learning a programming language can be difficult because they are extremely sensitive to even the slightest of typos.
Most programming languages are case-sensitive, have obscure syntax, and will refuse to run if you get so much as a
semicolon in the wrong place—or worse, leave one out. Further, most programming languages in use today are
based on English and their syntax cannot be localized.
In contrast, a well-done block language can eliminate syntax errors completely. You can still create a program which
does the wrong thing, but you cannot create one with the wrong syntax: the blocks just won't fit that way. Block
languages are more discoverable: you can see all the constructs and libraries of the language right in the list of
blocks. Further, blocks can be localized into any human language without changing the meaning of the programming
language.
Figure 1.1 - The Blockcode IDE in use
Block-based languages have a long history, with some of the prominent ones being Lego Mindstorms, Alice3D,
StarLogo, and especially Scratch. There are several tools for block-based programming on the web as well: Blockly,
AppInventor, Tynker, and many more.
The code in this chapter is loosely based on the open-source project Waterbear, which is not a language but a tool for
wrapping existing languages with a block-based syntax. Advantages of such a wrapper include the ones noted above:
eliminating syntax errors, visual display of available components, ease of localization. Additionally, visual code can
file:///C:/Users/Limited/Desktop/limited/500%20line%20or%20less/1.html
1/13
2018/2/22
500 Lines or Less | Blockcode: A visual programming toolkit
sometimes be easier to read and debug, and blocks can be used by pre-typing children. (We could even go further
and put icons on the blocks, either in conjunction with the text names or instead of them, to allow pre-literate children
to write programs, but we don't go that far in this example.)
The choice of turtle graphics for this language goes back to the Logo language, which was created specifically to
teach programming to children. Several of the block-based languages above include turtle graphics, and it is a small
enough domain to be able to capture in a tightly constrained project such as this.
If you would like to get a feel for what a block-based-language is like, you can experiment with the program that is
built in this chapter from author's GitHub repository.
Goals and Structure
I want to accomplish a couple of things with this code. First and foremost, I want to implement a block language for
turtle graphics, with which you can write code to create images through simple dragging-and-dropping of blocks,
using as simple a structure of HTML, CSS, and JavaScript as possible. Second, but still important, I want to show
how the blocks themselves can serve as a framework for other languages besides our mini turtle language.
To do this, we encapsulate everything that is specific to the turtle language into one file ( turtle.js ) that we can
easily swap with another file. Nothing else should be specific to the turtle language; the rest should just be about
handling the blocks ( blocks.js and menu.js ) or be generally useful web utilities ( util.js , drag.js ,
file.js ). That is the goal, although to maintain the small size of the project, some of those utilities are less
general-purpose and more specific to their use with the blocks.
One thing that struck me when writing a block language was that the language is its own IDE. You can't just code up
blocks in your favourite text editor; the IDE has to be designed and developed in parallel with the block language.
This has some pros and cons. On the plus side, everyone will use a consistent environment and there is no room for
religious wars about what editor to use. On the downside, it can be a huge distraction from building the block
language itself.
The Nature of Scripts
A Blockcode script, like a script in any language (whether block- or text-based), is a sequence of operations to be
followed. In the case of Blockcode the script consists of HTML elements which are iterated over, and which are each
associated with a particular JavaScript function which will be run when that block's turn comes. Some blocks can
contain (and are responsible for running) other blocks, and some blocks can contain numeric arguments which are
passed to the functions.
In most (text-based) languages, a script goes through several stages: a lexer converts the text into recognized
tokens, a parser organizes the tokens into an abstract syntax tree, then depending on the language the program may
be compiled into machine code or fed into an interpreter. That's a simplification; there can be more steps. For
Blockcode, the layout of the blocks in the script area already represents our abstract syntax tree, so we don't have to
go through the lexing and parsing stages. We use the Visitor pattern to iterate over those blocks and call predefined
JavaScript functions associated with each block to run the program.
There is nothing stopping us from adding additional stages to be more like a traditional language. Instead of simply
calling associated JavaScript functions, we could replace turtle.js with a block language that emits byte codes
for a different virtual machine, or even C++ code for a compiler. Block languages exist (as part of the Waterbear
project) for generating Java robotics code, for programming Arduino, and for scripting Minecraft running on Raspberry
Pi.
Web Applications
In order to make the tool available to the widest possible audience, it is web-native. It's written in HTML, CSS, and
JavaScript, so it should work in most browsers and platforms.
Modern web browsers are powerful platforms, with a rich set of tools for building great apps. If something about the
implementation became too complex, I took that as a sign that I wasn't doing it "the web way" and, where possible,
tried to re-think how to better use the browser tools.
An important difference between web applications and traditional desktop or server applications is the lack of a
main() or other entry point. There is no explicit run loop because that is already built into the browser and implicit
on every web page. All our code will be parsed and executed on load, at which point we can register for events we
are interested in for interacting with the user. After the first run, all further interaction with our code will be through
file:///C:/Users/Limited/Desktop/limited/500%20line%20or%20less/1.html
2/13
2018/2/22
500 Lines or Less | Blockcode: A visual programming toolkit
callbacks we set up and register, whether we register those for events (like mouse movement), timeouts (fired with
the periodicity we specify), or frame handlers (called for each screen redraw, generally 60 frames per second). The
browser does not expose full-featured threads either (only shared-nothing web workers).
Stepping Through the Code
I've tried to follow some conventions and best practices throughout this project. Each JavaScript file is wrapped in a
function to avoid leaking variables into the global environment. If it needs to expose variables to other files it will
define a single global per file, based on the filename, with the exposed functions in it. This will be near the end of the
file, followed by any event handlers set by that file, so you can always glance at the end of a file to see what events it
handles and what functions it exposes.
The code style is procedural, not object-oriented or functional. We could do the same things in any of these
paradigms, but that would require more setup code and wrappers to impose on what exists already for the DOM.
Recent work on Custom Elements make it easier to work with the DOM in an OO way, and there has been a lot of
great writing on Functional JavaScript, but either would require a bit of shoe-horning, so it felt simpler to keep it
procedural.
There are eight source files in this project, but index.html and blocks.css are basic structure and style for the
app and won't be discussed. Two of the JavaScript files won't be discussed in any detail either: util.js contains
some helpers and serves as a bridge between different browser implementations—similar to a library like jQuery but
in less than 50 lines of code. file.js is a similar utility used for loading and saving files and serializing scripts.
These are the remaining files:
block.js is the abstract representation of a block-based language.
drag.js implements the key interaction of the language: allowing the user to drag blocks from a list of available
blocks (the "menu") to assemble them into a program (the "script").
menu.js has some helper code and is also responsible for actually running the user's program.
turtle.js defines the specifics of our block language (turtle graphics) and initializes its specific blocks. This is
the file that would be replaced in order to create a different block language.
blocks.js
Each block consists of a few HTML elements, styled with CSS, with some JavaScript event handlers for dragging-
and-dropping and modifying the input argument. The blocks.js file helps to create and manage these groupings of
elements as single objects. When a type of block is added to the block menu, it is associated with a JavaScript
function to implement the language, so each block in the script has to be able to find its associated function and call it
when the script runs.
Figure 1.2 - An example block
Blocks have two optional bits of structure. They can have a single numeric parameter (with a default value), and they
can be a container for other blocks. These are hard limits to work with, but would be relaxed in a larger system. In
Waterbear there are also expression blocks which can be passed in as parameters; multiple parameters of a variety
of types are supported. Here in the land of tight constraints we'll see what we can do with just one type of parameter.
Right
degrees
It's important to note that there is no real distinction between blocks in the menu and blocks in the script. Dragging
treats them slightly differently based on where they are being dragged from, and when we run a script it only looks at
file:///C:/Users/Limited/Desktop/limited/500%20line%20or%20less/1.html
3/13