Investigate integrating or replacing generators with Plop

[cannikin]

https://plopjs.com/

• Could this be a viable alternative to maintaining our own generator code?
• Can we alias plop inside of Redwood so we can still use yarn rw g?
• Can we generate both TS and JS files?
• Regarding Failures: If we need to generate multiple files and generation fails somewhere in the middle of writing those files to disk, can the already created files be undone?

/cc @kimadeline word on the street this is being used by some Microsoft tools! Do you have any inside scoop? 😃

[kimadeline]

😆

Did some sleuthing 🕵️‍♀️ First result that came up was Just for build tasks (still haven't figured out how to word my search queries to find out how it stands against Gulp), followed by the React components for Fluent UI. It's the design system for Microsoft 365/Office 365, and although it looks like it's mostly using Plop by proxy through Just, there seems to be plans to use Plop directly.

</🕵️‍♀️>

[cannikin]

Thank you for the investigation! 🔍

After suffering through our existing generator code (don't know who built that originally...<cough, cough, nervous glance>) do you have any opinion about whether we should maybe swap out to something like plop (and contributing back any enhancements we might come up with)?

[kimadeline]

Not a Microsoft tool, but it's used by Gatsby too!

So far it looks like a "maybe yes" answer to your first bullet point, so I think it's worth investigating, at least to see if it becomes a definitive yes or a hell no 🙅‍♀️ and address your 2 other questions at the same time 😃

[Tobbe]

Exciting!

• Could this be a viable alternative to maintaining our own generator code?

Looks like it could be!

• Can we alias plop inside of Redwood so we can still use yarn rw g?

If by no other way one can always use child_process.spawn or .exec
There is also this https://github.com/plopjs/node-plop/

• Can we generate both TS and JS files?

From their own web page:
"Plop can generate any type of code! Use plop to generate HTML, CSS, JavaScript, CSharp, Objective C, Selenium tests, microservices... anything!"

So I guess so 😉

I think the first thing to figure out would be how to interface with or wrap Plop. I'm sure it can do what we need, I wonder how much extra we would get compared to lodash's templates...

Especially from a testability standpoint using the embedded version (node-plop) would probably be easier. But then we're in very undocumented territory, or, as they put it themselves: "Thar be dragons." 😛

[peterp]

Can we generate both TS and JS files?

We would want the source for either JS or TS be the same template since we don't want to manage multiple templates. It would be great if that's supported, but if it isn't we could still generate .ts files and then use babel and prettier to convert them to .js files.

[Tobbe]

I'm pretty sure there is no magic way to have it generate both kinds of files.

It would probably have to look something like this (contrived example, because, aren't they always? 😆)

{{#if ts}}
interface Length {
    length: number;
}
{{/if}}

function lengthTimesFive{{#if ts}}<T extends Length>{{/if}}(arg{{#if ts}}: T{{/if}}) {
   return arg.length * 5;
}

With ts === true we'd get this

interface Length {
    length: number;
}

function lengthTimesFive<T extends Length>(arg: T) {
   return arg.length * 5;
}

And for JS (ts === false) we'd get this

function lengthTimesFive(arg) {
   return arg.length * 5;
}

[peterp]

@Tobbe I think the approach that we're currently using, which is to generate the typescript template and then to convert it to a javascript file with babel + prettier works fairly effectively without having to use any branching logic.

[kimadeline]

Regarding Failures: If we need to generate multiple files and generation fails somewhere in the middle of writing those files to disk, can the already created files be undone?

Doesn't look like there's an "undo" method (there's abortOnFail but it's for queued actions, not what has already executed), so some way to do it would probably be to build a custom action (documentation) that saves (or makes a copy) of the initial state, and overwrites the corrupted state if the custom action failed somehow.

[peterp]

@kimadeline Thanks for researching this, I think your suggestion for adding a custom action is a super reasonable approach.

I would like to write down a set of first steps; would you be interested in contributing this?

1. Create a new @redwoodjs/generators package: Right now we have our generators inside of the CLI package, which is fine since that's where they're used, but they're somewhat hidden and probably a place that people will want to contribute to very frequently. I would also like to programatically execute the generators without having to call a CLI command. (I can setup this step.)
2. Create a typescript template for yarn rw g auth netlify: I think this template is fairly complicated since it modifies a bunch of lines of different files and adds/ installs dependencies.

@cannikin Does that sound like a reasonable way of testing plop?

[Tobbe]

I would also like to programatically execute the generators without having to call a CLI command.

Absolutely agree this is the best approach if possible. Would probably mean we'll have to go with https://github.com/plopjs/node-plop

If that's not a route we want to go down, it's actually not too bad executing yarn commands with node. I've recently done it for when I set up cypress ci execution for a project of mine

[noire-munich]

I would also like to programatically execute the generators without having to call a CLI command.

Can you elaborate, @peterp & @Tobbe ?
I must admit that I have fired all the generators at the beginning of the project but didn't feel comfortable working with the results, so I moved to my own components and kept only a very few selected files. Could we keep the choice on using generated code, maybe based on a config value?
Opened to discussion :)

[peterp]

@noire-munich It totally would be opt in only, not much will change, but I think for a testability / tooling development / exception handling standpoint that not explicitly having to us the CLI is a bit easier.

[kimadeline]

@peterp Sounds like a good starting point, count me in! I'd be happy to give it a try after we're sure that's the way we want to go to test it, and/or your setup pass 😬

[peterp]

@kimadeline Woot! Thanks! It is OK for me to assign you to this issue in the meantime? - I should have the "generators" package setup by tomorrow, and then I'll improve the granularity of "step 2" that I outlined above.

[peterp]

@Tobbe Thanks for your input and support on this issue 🙇‍♂️

[kimadeline]

@peterp Of course, sounds good! 😄

[cannikin]

@cannikin Does that sound like a reasonable way of testing plop?

Yep sounds great to me!

[jtoar]

Bison, a fellow fullstack-to-jamstack solution (that uses Cells! 🚀), uses Hygen, which may also be worth considering. @cball what made you choose Hygen over Plop?

[cball]

@jtoar we've used Hygen for years on pretty much every project we can. I've looked into Plop previously and the two tools are similar enough we didn't feel the need to switch.

That said, there are some things Hygen offers that I prefer:

• uses a folder/file convention for defining generators and their sub actions. No other modifications needed.
• when you add a new action, the action and any associated templates are in a single file (Hygen uses frontmatter at the top of the ejs file). This is a bit more portable, but that aspect may not matter much with a project like Redwood.
• uses ejs instead of handlebars for the templates

Overall though, I feel like it's mostly a syntax choice and they accomplish the same thing. Plop seems to have a bit more community awareness.

[thedavidprice]

Update: in conversation today with Peter P., he gave Plop a 👎

We are not specifically unhappy with the current Redwood generators. However, there are two topics that keep coming up:

1. Extensibility: in the future, how to support BYOG
2. Maintainability: generator code is expanding rapidly, and things like tests are becoming 😬

Peter mentioned wanting to first explore improving the structure and/or refactor before looking into an existing library like Plop or Hygen.

TBD

[jtoar]

There's an opportunity to refactor the generators (possibly into a separate package) during the RC. Since this issue was mainly about using an ecosystem library, and since that's been delayed till after we've refactored, I'll close this for now.