Creating own language made easy

Creating own language
made easy
Ingvar Stepanyan
@RReverser

Everything sucks – human version

Everything sucks – developer version

Not so scary magic

JS

JS

JS
Parser
Generator

Parser generators


jison Bison in javascript, used by Coffeescript



PEG.js parser generator for JavaScript based on the parsing expression grammar formalism



OMeta/JS (source) metacompiler for many languages to many targets, including js.



languagejs - PEG parser written in JavaScript with first class errors



Canopy Self-hosting PEG parser compiler for JavaScript, influenced by Ruby parser generators such as Treetop and
Citrus



JS/CC LALR(1) parser generator



jsparse PEG by Grandmaster Chris Double



ReParse parser combinator library for Javascript like Haskell's Parsec



p4js Monadic parser library for JavaScript



JSGLR Scannerless, Generalized Left-to-right Rightmost (SGLR) derivation parser for JavaScript



antlr has a javascript target



Cruiser.Parse LL(k) parser

Parser generators
Top-down (Jison)

Bottom-up (PEG.js)
start
= additive

%left "+"
%left "*"

additive
= left:multiplicative "+" right:additive { return
left + right; }
/ multiplicative

%start program;

multiplicative
= left:primary "*" right:multiplicative { return
left * right; }
/ primary

program
: expression { return $$ }
;

primary
= integer
/ "(" additive:additive ")" { return additive; }
integer "integer"
= digits:[0-9]+ { return parseInt(digits.join(""),
10); }

%%

expression
: expression "+" expression -> $1 + $3
| expression "*" expression -> $1 * $3
| "(" expression ")" -> $2
| NUMBER -> $1
;

Choice ordering
Top-down (Jison)

Bottom-up (PEG.js)
start
= additive

%left "+"
%left "*"

additive
left + right; }
/ multiplicative

%start program;

multiplicative
left * right; }
/ primary

program
;

primary
= integer
integer "integer"
10); }

%%

expression
| "(" expression ")" -> $2
| NUMBER -> $1
;

Ambiguity
if a then (if b then s) else s2
or
if a then if b then s else s2:
if a then (if b then s else s2)

Top-down (Jison)

Bottom-up (PEG.js)
start
= additive

%left "+"
%left "*"

additive
left + right; }
/ multiplicative

%start program;

multiplicative
left * right; }
/ primary

program
;

primary
= integer
integer "integer"
10); }

%%

expression
| "(" expression ")" -> $2
| NUMBER -> $1
;

Left recursion
x=1-2-3
Top-down (Jison)

Bottom-up (PEG.js)
start
= additive

%left "+"
%left "*"

additive
left + right; }
/ multiplicative

%start program;

multiplicative
left * right; }
/ primary

program
;

primary
= integer
integer "integer"
10); }

%%

expression
| "(" expression ")" -> $2
| NUMBER -> $1
;

Left recursion
x=1-2-3
Top-down (Jison)

Bottom-up (PEG.js)
[

[

"x",
"=",
[
[
"1",
"-",
"2"
],
"-",
"3"
]

"x",
"=",
[
"1",
"-",
[
"2",
"-",
"3"
]
]
]

]

Summary choice
Top-down (Jison)

Bottom-up (PEG.js)
start
= additive

%left "+"
%left "*"

additive
left + right; }
/ multiplicative

%start program;

multiplicative
left * right; }
/ primary

program
;

primary
= integer
integer "integer"
10); }

%%

expression
| "(" expression ")" -> $2
| NUMBER -> $1
;

Jison syntax: helpers
%{

var scope = {};
%}
…

Jison syntax: lexer
…

%lex
%%
s+

/* skip whitespace */

[A-Za-z_]w+

return 'ID';

d+

return ‘NUMBER’;

[+*;=]

return yytext;

<<EOF>>

return 'EOF';

/lex
…

Jison syntax: operator precedence
…

%left ';‘
%right ‘=‘
%left ‘+’
%left ‘*’

…
/*
“x=a*b+c”
-> assign(“x”, “a*b+c”)
-> assign(“x”, add(“a*b”, “c”))
-> assign(“x”, add(mul(“a”, “b”), “c”))
*/

Jison syntax: rules
%start program
program
: stmt* EOF { return $1 }
;
stmt
: expr ‘;’ -> $1
;
expr
| NUMBER -> $1
;

Methods
 string concatenation
 building AST object + escodegen (http://github.com/Constellation/escodegen)
 using SourceNode from source-map (https://github.com/mozilla/source-map)

SourceNode
 new SourceNode(line, column, filename, jsChunk)
 line, column – position in original file
 filename – name of original file
 jsChunk – JavaScript code string, another SourceNode instance or array of those

Resulting stack

JS

JS

JS
Jison
sourcemap

Creating own language made easy

More Related Content

What's hot

Similar to Creating own language made easy

More from Ingvar Stepanyan

Recently uploaded

Creating own language made easy