-
Notifications
You must be signed in to change notification settings - Fork 42
Semantic templating
The following documents ideas around nesting grammars inside Sysl. This is to allow code generators written in Sysl that have the look-n-feel of the target language. Think of it as templating on steroids.
Extant code generators are written in Python.
# Raw
def GenGoIntf(t):
print 'type ' + t.name + ' interface {'
for f in t.fields:
print '\t' + goCase(f.name) + '() ' + goTypes[f.type]
print '}'
# With helpers
def GenGoIntf(t, w):
with w.Block('type {} interface', t.name):
for f in t.fields:
w.Line('{}() {}', goCase(f.name), goTypes[f.type])Straight port from Python to Go.
func GenGoIntf(t sysl.Type) {
fmt.Printf("type %s interface {\n", t.Name)
for _, f := range t.Fields {
fmt.Printf("\t%s() %s\n", goCase(t.Name), goTypes[t.Type])
}
fmt.Println("}")
}Decouple the single process into first generating an AST, then transcribing the AST into source code.
func GenGoIntf(t sysl.Type) *golang.InterfaceDecl {
typ := &golang.InterfaceDecl{Name: t.Name}
for _, f := range t.Fields {
methodSpec := &golang.MethodSpec{
Name: goCase(t.Name),
Type: goTypes[t.Type],
}
typ.MethodSpec = append(typ.MethodSpec, methodSpec)
}
return typ
}The AST can then write itself to the target source file. This is just one approach. (Also possible are a standalone function or a method of a GoSourceWriter type. The latter would permit some degree of parameterisation, such as logging the code-generation process more or less noisily. It could also be used to specify instrumented variants of the output code, though a better idea in this case would be to do an AST-to-AST transform.)
// Write writes i to w as Go source code.
func (i *golang.InterfaceDecl) Write(w io.Writer) {
fmt.Fprintf(w, "type %s interface {\n", i.Name)
for _, f := range i.MethodSpec {
fmt.Fprintf(w, "\t%s() %s\n", i.Name, i.Type)
}
}Port the phase 1 solution from Go to Sysl.
- This will likely entail moving
sysl.Typefrom a protobuf schema to a sysl schema.
!view GenGoIntf(t <: sysl.Type) -> golang.InterfaceDecl:
t -> {
Name: .Name
MethodSpec: .MethodSpec => {
Name: goCase(.Name),
Type: goTypes(.Type),
}
}
Define a target-language AST via a grammar.
- The view code remains the same.
- The only change is that
golang.InterfaceDeclwill be defined by a sysl-defined target-language grammar instead of a sysl type definition.
The following syntax is indicative only.
!grammar go:
InterfaceDecl:
type Ident<Name> interface {
MethodSpec*
}
MethodSpec:
Ident<Name> ( ) Ident<Type>
Block:
{
Stmt*
}
Ident = /[A-Za-z_]([A-Za-z_0-9])*/
Implement nested syntax for fixed target-language content.
- Using the sysl-defined grammar, allow constant expressions to be written as snippets of target-language syntax.
# Before
!view GenGoFooIntf() -> golang.InterfaceDecl:
t -> golang.InterfaceDecl{
Name: "Foo",
MethodSpec: [
golang.MethodSpec{Name: "x", Type: "int"},
golang.MethodSpec{Name: "y", Type: "int"},
]
}
# After
!view GenGoFooIntf() -> golang.InterfaceDecl:
t -> <!golang.InterfaceDecl
type Foo interface {
x() int
y() int
}
!>
Implement nested syntax for parameterised target-language content.
- Using the sysl-defined grammar, allow dynamic expressions to be written as snippets of target-language syntax with arbitrarily deep nesting of sysl transforms and target-language syntax.
!view GenGoIntf(t <: sysl.Type) -> golang.InterfaceDecl:
t -> <!golang.InterfaceDecl
type ${.Name} interface {
${.MethodSpec => <!golang.MethodSpec
${goCase(.Name)}() ${goTypes(.Type)}
!>}
}
!>
Type-inference on nested syntax.
!view GenGoIntf(t <: sysl.Type) -> golang.InterfaceDecl:
t -> <!
type ${.Name} interface {
${.MethodSpec => <!
${goCase(.Name)}() ${goTypes(.Type)}
!>}
}
!>
It is worth nothing that this example doesn't look very Go-like. In this case, it's because the bulk of the content is parameterised. Other examples will have more Go boilerplate, and the benefit will be more obvious in such cases. Also, syntax highlighting should make things more readable.
(The following section doesn't make a whole lot of sense. Don't burst a blood vessel trying to figure it out.)
Say we define a Sysl type as follows. This example defines the AST for a binary expression in Go.
Go [package="ast"]:
!type BinaryExpr:
X <: Expr
Op <: Token
Y <: Expr
We want to generate a Go struct that implements the above type. We want it to look something like the following:
type BinaryExpr struct {
X Expr
Op Token
Y Expr
}So, how do we get from Sysl to Go?
Getting Sysl to parse the original file produces a protobuf that JSON-serialises to something like the following (this is just from memory and is indicative, not definitive):
{
"Name": "Go",
"Attrs": {
"package": "myapp"
},
"Types": {
"BinaryExpr": {
"Attrs": {
"X": {
"Type": {
"Ref": "Expr"
}
},
"Op": {
"Type": {
"Ref": "Token"
}
},
"Y": {
"Type": {
"Ref": "Expr"
}
}
}
}
}
}From this, we want to generate a Go AST representing the second file, above.
&golang.File{
Name: "ast",
Decls: []*golang.Decl{
&golang.GenDecl{
Tok: "type",
Specs: &golang.TypeSpec{
Name: "BinaryExpr",
Type: &golang.StructType{
Struct: "struct",
Fields: &golang.FieldList{
List: []*golang.Field{
&golang.Field{
Names: []*golang.Ident{"X"},
Type: "Expr",
},
&golang.Field{
Names: []*golang.Ident{"Op"},
Type: "Token",
},
&golang.Field{
Names: []*golang.Ident{"Y"},
Type: "Expr",
},
},
},
},
},
},
},
}Where do the golang.* types come from? That's an interesting question. For now, let's just assume they have been handcrafted in advance.
Now we want to transform the parsed protobuf structure into a Go AST. The following code illustrates how we would do this in phase 1, as described above:
func ModuleToGoAst(module *sysl.Module) *golang.File {
return &golang.File{
Name: module.Attrs["package"].(string),
Decls: SyslTypeMapToGoDecls(module.Types),
}
}
func SyslTypeMapToGoDecls(typeMap map[string]*sysl.Type) []*golang.Decl {
decls := []*golang.Decl{}
for name, t := range typeMap {
decls = append(decls, SyslNameAndTypeToGoDecl(name, t))
}
return decls
}
func SyslNameAndTypeToGoDecl(name string, t *sysl.Type) {
return &golang.Decl{
Tok: "type",
&golang.TypeSpec{
Name: name,
Type: SyslTypeToGoStruct(t),
},
}
}
func SyslTypeToGoStruct(t *sysl.Type) *golang.StructType {
fields := []*golang.Field{}
for name, attr := range t.Attrs {
fields = append(fields, &Field{
Names: :[]*Ident{name},
Type: attr.Type.Ref,
})
}
return &golang.StructType{
Struct: "struct",
Fields: &golang.FieldList{
List: fields,
}
}
}Above, we assumed golang.* types were readily available, but rather than handcraft them, can't we generate them from Sysl types? Well, that's exactly what the above code is intended to do. In other words, this code is generating the code it depends on! This is something of a pickle, and we might indeed have to handcraft the original Go AST, given that Go is the language in which Sysl is implemented. Alternatively, we could use Go's ast package directly, but that would be no fun!