Controlling Flow

callbacks made are easy

Thursday, October 28, 2010

EventEmitters are Easy

Responding to events is a solved problem
(At least for JavaScripters)

Very similar to DOM coding thing.on("event", doSomething) easy.

Thursday, October 28, 2010

callbacks are hard

Most common complaint about nodejs:
Ew, callbacks? Ugly nasty nesting indented forever spaghetti code? Just to open a le?! YOUVE GOTTA BE FRICKIN KIDDING ME.

Variant:
Why doesnt this work? <link to severely broken code>
Thursday, October 28, 2010

It's worse than that

Most *objects* in NodeJS are Event
Emitters (http server/client, etc.) (posix API, DNS lookups, etc.)

Most low level *functions* take callbacks. Beyond "hello, world" it gets tricky.
Thursday, October 28, 2010

What's actually hard?

Doing a bunch of things in a specic order. Knowing when stuff is done. Handling failures. Breaking up functionality into parts
(innitely nested inline callbacks)

Thursday, October 28, 2010

Common Mistakes
Abandoning convention and consistency. Putting all callbacks inline. Using libraries without grokking them. Trying to make async code look sync.*
*controversial In my opinion, promises are a perfectly ne way to solve this problem. But they're complicated under the hood, and making async code look sync can cause weird expectations. Also, people like to talk about a "Promise" like it's one kind of thing, when it's actually a very general pattern.
Thursday, October 28, 2010

ow control libraries
There are approximately 7 gillion ow Everyone likes their own the best. Obvious solution: Write your own. Let's do that now.
Thursday, October 28, 2010

control libraries in the node.js ecosystem.

This is a learning exercise

The goal is not to write the ideal ow
control library.

The goal is simplicity and understanding,

with very little magic.

Please dont hold questions for the end. Please do try this at home.
Thursday, October 28, 2010

First Priority: A Really Cool Name

Be descriptive, but not much to describe. So minimal, you can write it in a slide show. http://github.com/isaacs/slide-ow-control npm install slide Hellz.Yeah.
Thursday, October 28, 2010

Dene Conventions
Two kinds of functions:
Actors: Take action Callbacks: Get results

Essentially the continuation pattern.

APIs already, and it's very exible.

Resulting code *looks* similar to bers, but is *much* simpler to implement.

Bonus: node works this way in the lowlevel

Thursday, October 28, 2010

Callbacks
Simple responders Must always be prepared to handle errors!
(That's why it's the rst argument.)

Often inline anonymous, but not always. Can trap and call other callbacks with
Thursday, October 28, 2010

modied data, or to pass errors upwards.

Actors
Last argument is a callback. If any error occurs, and can't be handled,
pass it to the callback and return.

Must not throw. Return value ignored. return x ==> return cb(null, x) throw er ==> return cb(er)
Thursday, October 28, 2010

Actor Example
function actor (some, args, cb) { // last argument is callback // optional args: if (!cb && typeof(args) === "function") cb = args, args = [] // do something, and then: if (failed) cb(new Error( "failed!")) else cb(null, optionalData)

Thursday, October 28, 2010

Actor Example
// return true if a path is either // a symlink or a directory. function isLinkOrDir (path, cb) { fs.lstat(path, function (er, s) { if (er) return cb(er) return cb(null, s.isDirectory() || s.isSymbolicLink()) }) }

Thursday, October 28, 2010

Actor Example
// return true if a path is either // a symlink or a directory. function isLinkOrDir (path, cb) { fs.lstat(path, function (er, s) { if (er) return cb(er) return cb(null, s.isDirectory() || s.isSymbolicLink()) }) }

Thursday, October 28, 2010

Actor Example
// return true if a path is either // a symlink or a directory. function isLinkOrDir (path, cb) { fs.lstat(path, function (er, s) { if (er) return cb(er) return cb(null, s.isDirectory() || s.isSymbolicLink()) }) }

Thursday, October 28, 2010

Actor Example
// return true if a path is either // a symlink or a directory. function isLinkOrDir (path, cb) { fs.lstat(path, function (er, s) { if (er) return cb(er) return cb(null, s.isDirectory() || s.isSymbolicLink()) }) }

Thursday, October 28, 2010

Actor Composition
// return true if a path is either // a symlink or a directory, and also // ends in ".bak" function isLinkDirBak (path, cb) { return isLinkOrDir(path, function (er, ld) { return cb(er, ld && path.substr(-4) === ".bak") }) }

Thursday, October 28, 2010

Actor Composition
// return true if a path is either // a symlink or a directory, and also // ends in ".bak" function isLinkDirBak (path, cb) { return isLinkOrDir(path, function (er, ld) { return cb(er, ld && path.substr(-4) === ".bak") }) }

Thursday, October 28, 2010

Actor Composition
// return true if a path is either // a symlink or a directory, and also // ends in ".bak" function isLinkDirBak (path, cb) { return isLinkOrDir(path, function (er, ld) { return cb(er, ld && path.substr(-4) === ".bak") }) }

Thursday, October 28, 2010

usecase: asyncMap
I have a list of 10 les, and need to read all I have a dozen URLs, and need to fetch
of them, and then continue when they're all done. them all, and then continue when they're all done. a message to all of them, and then continue when that's done.

I have 4 connected users, and need to send

Thursday, October 28, 2010

usecase: asyncMap
I have a list of n things, and I need to

dosomething with all of them, in parallel, and get the results once they're all complete.

Thursday, October 28, 2010

function asyncMap (list, fn, cb_) { var n = list.length , results = [] , errState = null function cb (er, data) { if (errState) return if (er) return cb(errState = er) results.push(data) if (-- n === 0) return cb_(null, results) } list.forEach(function (l) { fn(l, cb) }) }
Thursday, October 28, 2010

function asyncMap (list, fn, cb_) { var n = list.length , results = [] , errState = null function cb (er, data) { if (errState) return if (er) return cb(errState = er) results.push(data) if (-- n === 0) return cb_(null, results) } list.forEach(function (l) { fn(l, cb) }) }
Thursday, October 28, 2010

function asyncMap (list, fn, cb_) { var n = list.length , results = [] , errState = null function cb (er, data) { if (errState) return if (er) return cb(errState = er) results.push(data) if (-- n === 0) return cb_(null, results) } list.forEach(function (l) { fn(l, cb) }) }
Thursday, October 28, 2010

function asyncMap (list, fn, cb_) { var n = list.length , results = [] , errState = null function cb (er, data) { if (errState) return if (er) return cb(errState = er) results.push(data) if (-- n === 0) return cb_(null, results) } list.forEach(function (l) { fn(l, cb) }) }
Thursday, October 28, 2010

usecase: asyncMap
function writeFiles (files, what, cb) { asyncMap( files , function (f, cb) { fs.writeFile(f,what,cb) } , cb ) } writeFiles([my,file,list], "foo", cb)

Thursday, October 28, 2010

asyncMap
note that asyncMap itself is an Actor
function, so you can asyncMap your asyncMaps, dawg. matter, but what if it does?

This implementation is ne if order doesn't

Thursday, October 28, 2010

asyncMap - ordered
close over the array index in the generated
cb function.

match up results to their original index.

Thursday, October 28, 2010

function asyncMap (list, fn, cb_) { var n = list.length , results = [] , errState = null function cbGen (i) { return function cb (er, data) { if (errState) return if (er) return cb(errState = er) results[i] = data if (-- n === 0) return cb_(null, results) }} list.forEach(function (l, i) { fn(l, cbGen(i)) })
Thursday, October 28, 2010

function asyncMap (list, fn, cb_) { var n = list.length , results = [] , errState = null function cbGen (i) { return function cb (er, data) { if (errState) return if (er) return cb(errState = er) results[i] = data if (-- n === 0) return cb_(null, results) }} list.forEach(function (l, i) { fn(l, cbGen(i)) })
Thursday, October 28, 2010

usecase: chain
I have to do a bunch of things, in order. Get If anything fails, do not continue.

db credentials out of a le, read the data from the db, write that data to another le.

Thursday, October 28, 2010

function chain (things, cb) { ;(function LOOP (i, len) { if (i >= len) return cb() things[i](function (er) { if (er) return cb(er) LOOP(i + 1, len) }) })(0, things.length) }

Thursday, October 28, 2010

function chain (things, cb) { ;(function LOOP (i, len) { if (i >= len) return cb() things[i](function (er) { if (er) return cb(er) LOOP(i + 1, len) }) })(0, things.length) }

Thursday, October 28, 2010

function chain (things, cb) { ;(function LOOP (i, len) { if (i >= len) return cb() things[i](function (er) { if (er) return cb(er) LOOP(i + 1, len) }) })(0, things.length) }

Thursday, October 28, 2010

function chain (things, cb) { ;(function LOOP (i, len) { if (i >= len) return cb() things[i](function (er) { if (er) return cb(er) LOOP(i + 1, len) }) })(0, things.length) }

Thursday, October 28, 2010

usecase: chain
Still have to provide an array of functions,
which is a lot of boilerplate, and a pita if your functions take args "function (cb){blah(a,b,c,cb)}"

Results are discarded, which is a bit lame. No way to branch.

Thursday, October 28, 2010

reducing boilerplate
convert an array of [fn, args] to an actor
that takes no arguments (except cb) our use-case.

A bit like Function#bind, but tailored for bindActor(obj, "method", a, b, c)

bindActor(fn, a, b, c) bindActor(obj, fn, a, b, c)

Thursday, October 28, 2010

function bindActor () { var args = Array.prototype.slice.call (arguments) // jswtf. , obj = null , fn if (typeof args[0] === "object") { obj = args.shift() fn = args.shift() if (typeof fn === "string") fn = obj[ fn ] } else fn = args.shift() return function (cb) { fn.apply(obj, args.concat(cb)) } }
Thursday, October 28, 2010

function bindActor () { var args = Array.prototype.slice.call (arguments) // jswtf. , obj = null , fn if (typeof args[0] === "object") { obj = args.shift() fn = args.shift() if (typeof fn === "string") fn = obj[ fn ] } else fn = args.shift() return function (cb) { fn.apply(obj, args.concat(cb)) } }
Thursday, October 28, 2010

function bindActor () { var args = Array.prototype.slice.call (arguments) // jswtf. , obj = null , fn if (typeof args[0] === "object") { obj = args.shift() fn = args.shift() if (typeof fn === "string") fn = obj[ fn ] } else fn = args.shift() return function (cb) { fn.apply(obj, args.concat(cb)) } }
Thursday, October 28, 2010

function bindActor () { var args = Array.prototype.slice.call (arguments) // jswtf. , obj = null , fn if (typeof args[0] === "object") { obj = args.shift() fn = args.shift() if (typeof fn === "string") fn = obj[ fn ] } else fn = args.shift() return function (cb) { fn.apply(obj, args.concat(cb)) } }
Thursday, October 28, 2010

bindActor
Some obvious areas for improvement. They wouldn't t on a slide. Left as an exercise for the reader.

Thursday, October 28, 2010

function chain (things, cb) { ;(function LOOP (i, len) { if (i >= len) return cb() if (Array.isArray(things[i])) things[i] = bindActor.apply (null, things[i]) things[i](function (er) { if (er) return cb(er) LOOP(i + 1, len) }) })(0, things.length) }

Thursday, October 28, 2010

function chain (things, cb) { ;(function LOOP (i, len) { if (i >= len) return cb() if (Array.isArray(things[i])) things[i] = bindActor.apply (null, things[i]) things[i](function (er) { if (er) return cb(er) LOOP(i + 1, len) }) })(0, things.length) }

Thursday, October 28, 2010

chain: branching
Skip over falsey arguments chain([ doThing && [thing,a,b,c]
, isFoo && [doFoo, "foo"] , subChain && [chain, [one, two]] ], cb)

Thursday, October 28, 2010

function chain (things, cb) { ;(function LOOP (i, len) { if (i >= len) return cb() if (Array.isArray(things[i])) things[i] = bindActor.apply (null, things[i]) if (!things[i]) return LOOP(i + 1, len) things[i](function (er) { if (er) return cb(er) LOOP(i + 1, len) }) })(0, things.length) }

Thursday, October 28, 2010

chain: tracking results

Supply an array to keep the results in. If you don't care, don't worry about it. Last result is always in results[results.length - 1] Just for kicks, let's also treat chain.rst and
chain.last as placeholders for the rst/last result up until that point.

Thursday, October 28, 2010

chain.first = {} ; chain.last = {} function chain (things, res, cb) { if (!cb) cb = res , res = [] ;(function LOOP (i, len) { if (i >= len) return cb(null,res) if (Array.isArray(things[i])) things[i] = bindActor.apply(null, things[i].map(function(i){ return (i===chain.first) ? res[0] : (i===chain.last) ? res[res.length - 1] : i })) if (!things[i]) return LOOP(i + 1, len) things[i](function (er, data) { res.push(er || data) if (er) return cb(er, res) LOOP(i + 1, len) }) })(0, things.length) }
Thursday, October 28, 2010

chain.first = {} ; chain.last = {} function chain (things, res, cb) { if (!cb) cb = res , res = [] ;(function LOOP (i, len) { if (i >= len) return cb(null,res) if (Array.isArray(things[i])) things[i] = bindActor.apply(null, things[i].map(function(i){ return (i===chain.first) ? res[0] : (i===chain.last) ? res[res.length - 1] : i })) if (!things[i]) return LOOP(i + 1, len) things[i](function (er, data) { res.push(er || data) if (er) return cb(er, res) LOOP(i + 1, len) }) })(0, things.length) }
Thursday, October 28, 2010

chain.first = {} ; chain.last = {} function chain (things, res, cb) { if (!cb) cb = res , res = [] ;(function LOOP (i, len) { if (i >= len) return cb(null,res) if (Array.isArray(things[i])) things[i] = bindActor.apply(null, things[i].map(function(i){ return (i===chain.first) ? res[0] : (i===chain.last) ? res[res.length - 1] : i })) if (!things[i]) return LOOP(i + 1, len) things[i](function (er, data) { res.push(er || data) if (er) return cb(er, res) LOOP(i + 1, len) }) })(0, things.length) }
Thursday, October 28, 2010

chain.first = {} ; chain.last = {} function chain (things, res, cb) { if (!cb) cb = res , res = [] ;(function LOOP (i, len) { if (i >= len) return cb(null,res) if (Array.isArray(things[i])) things[i] = bindActor.apply(null, things[i].map(function(i){ return (i===chain.first) ? res[0] : (i===chain.last) ? res[res.length - 1] : i })) if (!things[i]) return LOOP(i + 1, len) things[i](function (er, data) { res.push(er || data) if (er) return cb(er, res) LOOP(i + 1, len) }) })(0, things.length) }
Thursday, October 28, 2010

chain.first = {} ; chain.last = {} function chain (things, res, cb) { if (!cb) cb = res , res = [] ;(function LOOP (i, len) { if (i >= len) return cb(null,res) if (Array.isArray(things[i])) things[i] = bindActor.apply(null, things[i].map(function(i){ return (i===chain.first) ? res[0] : (i===chain.last) ? res[res.length - 1] : i })) if (!things[i]) return LOOP(i + 1, len) things[i](function (er, data) { res.push(er || data) if (er) return cb(er, res) LOOP(i + 1, len) }) })(0, things.length) }
Thursday, October 28, 2010

chain.first = {} ; chain.last = {} function chain (things, res, cb) { if (!cb) cb = res , res = [] ;(function LOOP (i, len) { if (i >= len) return cb(null,res) if (Array.isArray(things[i])) things[i] = bindActor.apply(null, things[i].map(function(i){ return (i===chain.first) ? res[0] : (i===chain.last) ? res[res.length - 1] : i })) if (!things[i]) return LOOP(i + 1, len) things[i](function (er, data) { res.push(er || data) if (er) return cb(er, res) LOOP(i + 1, len) Ok, this can't get any }) bigger or it won't t. })(0, things.length) }
Thursday, October 28, 2010

Non-trivial Use Case

Read number les in a directory Add the results together Ping a web service with the result Write the response to a le Delete the number les
Thursday, October 28, 2010

var chain = require("./chain.js") , asyncMap = require("./async-map.js") function myProgram (cb) { var res = [], last = chain.last , first = chain.first chain ( [ [fs, "readdir", "the-directory"] , [readFiles, "the-directory", last] , [sum, last] , [ping, "POST", "example.com", 80 , "/foo", last] , [fs, "writeFile", "result.txt", last] , [rmFiles, "./the-directory", first] ] , res , cb ) )
Thursday, October 28, 2010

var chain = require("./chain.js") , asyncMap = require("./async-map.js") function myProgram (cb) { var res = [], last = chain.last , first = chain.first chain ( [ [fs, "readdir", "the-directory"] , [readFiles, "the-directory", last] , [sum, last] , [ping, "POST", "example.com", 80 , "/foo", last] , [fs, "writeFile", "result.txt", last] , [rmFiles, "./the-directory", first] ] , res , cb ) )
Thursday, October 28, 2010

var chain = require("./chain.js") , asyncMap = require("./async-map.js") function myProgram (cb) { var res = [], last = chain.last , first = chain.first chain ( [ [fs, "readdir", "the-directory"] , [readFiles, "the-directory", last] , [sum, last] , [ping, "POST", "example.com", 80 , "/foo", last] , [fs, "writeFile", "result.txt", last] , [rmFiles, "./the-directory", first] ] , res , cb ) )
Thursday, October 28, 2010

var chain = require("./chain.js") , asyncMap = require("./async-map.js") function myProgram (cb) { var res = [], last = chain.last , first = chain.first chain ( [ [fs, "readdir", "the-directory"] , [readFiles, "the-directory", last] , [sum, last] , [ping, "POST", "example.com", 80 , "/foo", last] , [fs, "writeFile", "result.txt", last] , [rmFiles, "./the-directory", first] ] , res , cb ) )
Thursday, October 28, 2010

var chain = require("./chain.js") , asyncMap = require("./async-map.js") function myProgram (cb) { var res = [], last = chain.last , first = chain.first chain ( [ [fs, "readdir", "the-directory"] , [readFiles, "the-directory", last] , [sum, last] , [ping, "POST", "example.com", 80 , "/foo", last] , [fs, "writeFile", "result.txt", last] , [rmFiles, "./the-directory", first] ] , res , cb ) )
Thursday, October 28, 2010

var chain = require("./chain.js") , asyncMap = require("./async-map.js") function myProgram (cb) { var res = [], last = chain.last , first = chain.first chain ( [ [fs, "readdir", "the-directory"] , [readFiles, "the-directory", last] , [sum, last] , [ping, "POST", "example.com", 80 , "/foo", last] , [fs, "writeFile", "result.txt", last] , [rmFiles, "./the-directory", first] ] , res , cb ) )
Thursday, October 28, 2010

var chain = require("./chain.js") , asyncMap = require("./async-map.js") function myProgram (cb) { var res = [], last = chain.last , first = chain.first chain ( [ [fs, "readdir", "the-directory"] , [readFiles, "the-directory", last] , [sum, last] , [ping, "POST", "example.com", 80 , "/foo", last] , [fs, "writeFile", "result.txt", last] , [rmFiles, "./the-directory", first] ] , res , cb ) )
Thursday, October 28, 2010

Convention Prots
Consistent API from top to bottom. Sneak in at any point to inject functionality.
(testable, reusable, etc.)

When ruby and python users whine, you

can smile condescendingly.

Thursday, October 28, 2010