Take Data Validation Seriously - Paul Milham, WildWorks

Take Data Validation
Seriously
Paul Milham, WildWorks

Outline
• Attacks
• Data Validation => Security
• Data Normalization => Stability
• Joi
• Tean
• Express Integration
• Hapi Integration
• Questions

Safety
• My job is to keep kids safe.
• How do we keep our application safe?
• Safe from what?

Attacks
• The web is full of jerks
• https://www.owasp.org/index.php/Category:Attack
• Read that for a bedtime horror story

SQL Injection
console.log(name); // paul
console.log(email); // '); DROP TABLE db.user; --
mysql.query(`INSERT INTO db.user (name, email) VALUES ('${name}', '$
{email}')`);

Shell Injection
console.log(pass); // "; rm -rf /"
require("child_process").exec(`
php -r "print crypt('${pass}','$1$rounds=1$salt$');"
`, (err, stdout, stderr) => {
});
// hopefully you're using containers

ReDOS
const msg = 'foo=bar' + ';'.repeat(65535) + 'domain=example.com';
console.time("regex");
console.log(msg.search(/;+$/));
console.timeEnd("regex"); // regex: 5854.071ms :(
• This is a sample vulnerability in tough cookie
• https://snyk.io/vuln/npm:tough-cookie:20160722
• Be careful of "evil" regex

Security
• It’s a scary world
• Security is important
• There’s a lot more than just those three

Validation
• Verify the shape of the data
• Malicious data can’t get in
• First line of defense

Simple Joi
"use strict";
const Joi = require("joi");
Joi.validate("srsly a string", Joi.string(), (err, value) => {
console.log(err); // null
console.log(value); // "srsly a string"
});

Joi Failure
Joi.validate(5, Joi.string(), (err, value) => {
console.log(err); // Error
console.log(value); // 5
});

Joi Schema
const schema = Joi.object().keys({
username: Joi.string().email({tldWhiteList: ["wildworks"]}).required(),
password: Joi.string().min(6).max(25).required(),
toolId: Joi.number().integer().required(),
});
Joi.validate({
username: "paul.milham@wildworks.com",
password: "justinbieber",
toolId: 9001,
}, schema, (err, value) => {
console.log(err);
console.log(value);
});

All In
username: Joi.string().email({tldWhiteList: ["wildworks"]}).required(),
});
Joi.validate({
username: "paul.milham@wildworks.com",
password: "justinbieber",
}, schema, (err, value) => {
console.log(err); // justinbieber is not allowed
});

All In
• Validating one field means validating them all
• Hard for devs to forget

Data Normalization
• Normalization is being a good citizen
• Normalization creates a contract with your
consumer
• Normalization goes a lot deeper than this (we'll
get to that later)

Joi Conversion
Joi.validate("1.916", Joi.number(), (err, value) => {
console.log(value.toFixed(1)); // 1.9 (No TypeError!)
});

Joi Defaults
Joi.validate(undefined, Joi.number().default(0), (err, value) => {
console.log(value.toFixed(1)); // 0.0 (No TypeError!)
});

Tean
• Declarative syntax (schemas are POJOs)
• Async
• Convert data into models
• https://www.npmjs.com/package/tean
• Tean should be considered experimental
• Note that custom validators were recently added to Joi

Tean Validation
// simple validation
tean.object({breakfast: "string"}, {breakfast: "bacon"}, (isValid,
result) => {
console.log(isValid); // true
console.log(result); // {breakfast: "bacon"}
});

Tean Failure
tean.object({breakfast: "string"}, {breakfast: null}, (isValid, result)
=> {
console.log(isValid); // false
console.log(result); // ["breakfast (null) is not a string"]
});

Tean Normalization
// optional parameters
tean.object({breakfast: “string(pancakes,waffles)?waffles”, addSyrup:
"bool?true"}, {breakfast: "pancakes"}, (isValid, result) => {
console.log(isValid); // true
console.log(result); // {breakfast: "pancakes", addSyrup: true}
// Note that the original object is not altered! Normalized and
validated data is passed into "result" in the callback
});

Model Mapping
tean.object(req.body.params, {
language: "language",
pageTitle: "string?",
users: ["unifiedUserUid", "?[]"],
}, (isValid, result) => {
});

Data Normalization
• Provides a friendly API
• Provides consistency and reliability
• Eliminates lots of common bugs

Express
• Everyone uses it!
• No built in validation!
• Too many exclamation points!
• https://expressjs.com/

Express + Joi
app.get('/:pageId', function (req, res) {
pageId: Joi.number().min(0).required(),
});
Joi.validate(req.params, schema, (err, value) => {
console.log(err);
req.params = value;
res.send(`Hello World! ${req.params.pageId}`);
});
});

Express + Tean
app.get('/:pageId', function (req, res) {
tean.object(req.body.params, {
page: "page",
}, (isValid, result) => {
res.send(`Hello World! ${result.pageId}`);
});
});

Problem
• We’re relying on the developer to remember to
validate
• This is a problem for maintenance and updates
• Middleware to the rescue!

Hapi
• Hapi isn't minimalist like Express
• Lots of options out of the box
• http://hapijs.com/

Hapi Validation
app.route({
method: "POST",
path: "/",
config: {
handler: (req, reply) => {
reply("hey!");
},
validate: {
payload: {
username: Joi.string().email().required(),
password: Joi.string().max(25).required(),
},
},
},
});

Take Away
• FORCE validation of data - an opt in system where
the developer can forget isn't good enough
• Make sure shape of data is acceptable
• No validation, no data
• This ensures malicious data does not enter your
application

Take Away
• FORCE normalization of data shape
• Data should always have a consistent shape
• This makes data access and usage reliable
• Eliminates lots of “stupid” bugs

On the Way Out
• Have you thought about data security on the way out?
• Mind blown!
• Prevent heartbleed (uninitialized buffer)
• Provide same stability contract for your client app (or
other consumer)

Take Data Validation Seriously - Paul Milham, WildWorks

More Related Content

What's hot

Similar to Take Data Validation Seriously - Paul Milham, WildWorks

More from NodejsFoundation

Recently uploaded

Take Data Validation Seriously - Paul Milham, WildWorks