jk transform

[squaremo]

Addresses #253.

• Add subcommand machinery
• Supply examples in help text
• Support literal JavaScript given as the script (-c, --exec)
• Support a file path given as the script (no flag)
• Support a module given as the script (-m, --module)
• Apply transform to each object in multidocs
• Balk at being expected to print to stdout if formats differ
  • Happily print YAML and YAMLStream (or JSON and JSONStream) to stdout
• [ ] accept input from stdin (but in which format?)
• Write files out in output directory
  • (scheme for deciding the paths based on input paths TBD)
• --inplace (or --overwrite or whatever)
  • balk if it would overwrite a file and --overwrite is not set

[squaremo]

A fun issue that has come up:

• part of the mooted design is that --inline (or rather, its absence), will cause any writes that would overwrite a file to be raised as errors. Currently, there's a boolean sent in std.write to say whether an overwrite is OK; if it's false, overwrites will be skipped (i.e., silently failed). One way to get to the mooted design is to have three modes for dealing with (potential) overwrites -- do the overwrite, skip it, or raise an error -- and make these available in the API. Another way would be to keep it as a boolean, but be able to change the meaning of false in the runtime. The latter design may be attractive because it can then be a global flag for the runtime, something like --overwrite=error|skip. (Or further: remove it as an option in the lib, and only control it from the runtime)

[dlespiau]

For more context, the overwrite fields is/was per-file because of the "node module template" use case: https://github.com/jkcfg/module-template/blob/master/module.js#L201

• if no README exists, create a basic one
• if a README already exist, don't touch it

That allows to both: bootstrap a project from scratch and get a full fledged node module ready to commit and regenerate the build files without touching certain files that may have been modified by the user.

The above seems like a nice enough use case to keep /me thinks.

[dlespiau]

This relates somewhat to not writing files if they are identical to what already exists on the fs: #189

[squaremo]

regenerate the build files without touching certain files that may have been modified by the user.

The above seems like a nice enough use case to keep /me thinks.

No argument here; I was hoping to keep all use cases supported as they are now. I think your use indicates a slight preference for controlling the overwrite behaviour entirely from the JavaScript -- what do you reckon?

[dlespiau]

Yes, I think so too, because it has to be decided by the script author on a per-file basis (in the case of generate). Not sure how this would transpose to transform though, having a global --in-place seems like a good idea too.

[squaremo]

I have realised that it is odd to have --stdout and for the default to be true -- that means, whenever you want to write to files, you pass --stdout=false. I think it might be better to default to be consistent with generate by outputting to files unless --stdout; and to default to raising an error before overwriting a file (ideally, before any files are written).

[squaremo]

Re --stdin: accepting values from stdin is tricky because we don't know what format it's in. We could have --stdin=yaml perhaps. Rather than making a decision, I'm punting it into the future.

[squaremo]

I think it might be better to default to be consistent with generate by outputting to files unless --stdout; and to default to raising an error before overwriting a file (ideally, before any files are written).

@dlespiau I also think that latter (raise an error rather than overwriting a file) would be a good default for jk generate -- I can do this in another PR, if you agree ..

[squaremo]

$ jk transform --help
Transform configuration files

Usage:
  jk transform <script> <file>... [flags]

Examples:

  running the default export of a module (or file) on each input document
    jk transform -o outputdir/ ./script.js ./inputdir/*.json
  running a function on each input, and printing the results to stdout
    jk transform --stdout -c '({ name: n, ...fields }) => ({ name: n + "dev", ...fields })' inputdir/*.yaml


Flags:
  -d, --emit-dependencies         emit script dependencies
  -c, --exec                      treat first argument as specifying literal JavaScript to execute
  -h, --help                      help for transform
  -i, --input-directory string    where to find files read in the script; if not set, the directory containing the script is used
  -m, --module                    treat first argument as specifying a module to load
  -o, --output-directory string   where to output generated files
      --overwrite                 allow input file(s) to be overwritten by output file(s); otherwise, an error will be thrown
  -p, --parameter name=value      set input parameters
  -f, --parameters filename       load parameters from a JSON or YAML file
      --stdout                    print the resulting values to stdout
  -v, --verbose                   verbose output

Wrinkle: --input-directory is a global flag, but doesn't have an interpretation here (because the individual files are named explicitly as arguments).

[dlespiau]

oooh, exciting, can't wait to play with it!