Skip to content

Commit

Permalink
Change syntax for top-level directives
Browse files Browse the repository at this point in the history
%name changes to:

#name example;

%left and %right change to:

#prec (
    #left a b
    #right c d
);
  • Loading branch information
nihei9 committed May 10, 2022
1 parent 0eb44f0 commit 3eb0e88
Show file tree
Hide file tree
Showing 22 changed files with 892 additions and 385 deletions.
40 changes: 22 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,12 @@ $ go install github.com/nihei9/vartan/cmd/vartan-go@latest
vartan uses BNF-like DSL to define your grammar. As an example, let's write a grammar that represents a simple expression.

```
%name expr
#name expr;
%left mul div
%left add sub
#prec (
#left mul div
#left add sub
);
expr
: expr add expr
Expand Down Expand Up @@ -234,7 +236,7 @@ exit status 1

### Grammar name

A grammar name `%name <Identifier>` is an identifier that represents a grammar name. For now, this identifier is used as a file name generated like _<grammar-name>\_parser.go_.
A grammar name `#name <Identifier>` is an identifier that represents a grammar name. For now, this identifier is used as a file name generated like _<grammar-name>\_parser.go_.

### Production rules

Expand Down Expand Up @@ -325,7 +327,7 @@ example 1:
Consider a grammar that accepts comma-separated list of integers. You can avoid including brackets and commas in an AST by specifying only the necessary symbols int the `#ast` directive parameters. Also, you can flatten an AST using `...` operator. `...` operator expands child nodes of a specified symbol.

```
%name example
#name example;
list
: '[' elems ']' #ast elems...
Expand Down Expand Up @@ -356,7 +358,7 @@ example 2:
Consider a grammar that accepts ternary-if expression (`<condition> ? <value if true> : <value if false>`). As this grammar contains two `int` symbols, you need to add labels to each symbol to distinguish them. A label consists of `@` + an identifier.

```
%name example
#name example;
if_expr
: id '?' int@true ':' int@false #ast id true false
Expand Down Expand Up @@ -405,7 +407,7 @@ An `#alias` directive aliases for a terminal symbol. You can use the alias in er
example:

```
%name example
#name example;
s
: id
Expand All @@ -424,7 +426,7 @@ When the parser shifts a terminal symbol having the `#push` directive, the curre
example:

```
%name example
#name example;
tag_pairs
: tag_pairs tag_pair
Expand Down Expand Up @@ -471,7 +473,7 @@ The parser doesn't shift a terminal symbol having a `#skip` directive. In other
example:

```
%name example
#name example;
s
: foo bar
Expand All @@ -497,7 +499,7 @@ foobar

### Operator precedence and associativity

`%left` and `%right` allow you to define precedence and associativiry of symbols. `%left`/`%right` each assign the left/right associativity to symbols.
`#left` and `#right` directives allow you to define precedence and associativiry of symbols. `#left`/`#right` each assign the left/right associativity to symbols.

When the right-most terminal symbol of an alternative has precedence or associativity defined explicitly, the alternative inherits its precedence and associativity.

Expand All @@ -506,11 +508,13 @@ When the right-most terminal symbol of an alternative has precedence or associat
The grammar for simple four arithmetic operations and assignment expression can be defined as follows:

```
%name example
#name example;
%left mul div
%left add sub
%right assign
#prec (
#left mul div
#left add sub
#right assign
);
expr
: expr add expr
Expand Down Expand Up @@ -541,11 +545,11 @@ assign
: '=';
```

`%left` and `%right` can appear multiple times, and the first symbols applied to will have the highest precedence. That is, `mul` and `div` have the highest precedence, and `assign` has the lowest precedence.
`#left` and `#right` can appear multiple times, and the first symbols applied to will have the highest precedence. That is, `mul` and `div` have the highest precedence, and `assign` has the lowest precedence.

⚠️ In many Yacc-like tools, the last symbols defined have the highest precedence. Not that in vartan, it is the opposite.

When you compile the above grammar, some conflicts occur. However, vartan can resolve the conflicts following `%left`, `%right`, and `#prec`.
When you compile the above grammar, some conflicts occur. However, vartan can resolve the conflicts following `#left`, `#right`, and `#prec`.

```
$ echo -n 'foo = bar = x * -1 / -2' | vartan parse example.json
Expand All @@ -572,7 +576,7 @@ expr
Incidentally, using no directives, you can define the above example as the following grammar:

```
%name example
#name example;
expr
: id assign expr
Expand Down Expand Up @@ -651,7 +655,7 @@ When a syntax error occurs, the parser pops states from a state stack. If a stat
Consider grammar of simple assignment statements.

```
%name example
#name example;
statements
: statements statement #ast statements... statement
Expand Down
42 changes: 26 additions & 16 deletions driver/conflict_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ func TestParserWithConflicts(t *testing.T) {
{
caption: "when a shift/reduce conflict occurred, we prioritize the shift action",
specSrc: `
%name test
#name test;
expr
: expr assign expr
Expand Down Expand Up @@ -48,7 +48,7 @@ assign: '=';
{
caption: "when a reduce/reduce conflict occurred, we prioritize the production defined earlier in the grammar",
specSrc: `
%name test
#name test;
s
: a
Expand All @@ -73,10 +73,12 @@ id: "[A-Za-z0-9_]+";
{
caption: "left associativities defined earlier in the grammar have higher precedence",
specSrc: `
%name test
#name test;
%left mul
%left add
#prec (
#left mul
#left add
);
expr
: expr add expr
Expand Down Expand Up @@ -120,9 +122,11 @@ mul: '*';
{
caption: "left associativities defined in the same line have the same precedence",
specSrc: `
%name test
#name test;
%left add sub
#prec (
#left add sub
);
expr
: expr add expr
Expand Down Expand Up @@ -166,10 +170,12 @@ sub: '-';
{
caption: "right associativities defined earlier in the grammar have higher precedence",
specSrc: `
%name test
#name test;
%right r1
%right r2
#prec (
#right r1
#right r2
);
expr
: expr r2 expr
Expand Down Expand Up @@ -218,9 +224,11 @@ id
{
caption: "right associativities defined in the same line have the same precedence",
specSrc: `
%name test
#name test;
%right r1 r2
#prec (
#right r1 r2
);
expr
: expr r2 expr
Expand Down Expand Up @@ -269,11 +277,13 @@ id
{
caption: "left and right associativities can be mixed",
specSrc: `
%name test
#name test;
%left mul div
%left add sub
%right assign
#prec (
#left mul div
#left add sub
#right assign
);
expr
: expr add expr
Expand Down
2 changes: 1 addition & 1 deletion driver/lac_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import (

func TestParserWithLAC(t *testing.T) {
specSrc := `
%name test
#name test;
S
: C C
Expand Down
Loading

0 comments on commit 3eb0e88

Please sign in to comment.