chore(web): replace lerna with npm workspaces and ts projects 🎡 #6525
Conversation
Fixes #6320. Part 1 of moving from lerna to a simpler, maintained monorepo solution, using TypeScript Projects and NPM Workspaces. There is more work to be done here. At this point, KeymanWeb builds and runs without errors, but the built file is substantially different, mostly in include order. Using TypeScript Projects, we move away from the need to run build scripts in various locations for almost all the Typescript modules. TODO: Embedded versions and tests have not been verified. TODO: developer/server is not yet verified. TODO: developer/js (needs a rename!) is not yet verified. TODO: Currently, the predictive-text folder needs refactoring to move the construction of the worker wrapper out of the Predictive Text build and into the final assembly of keymanweb.js (as it should be valid to run it as a separate .js anyway). TODO: Most of the `<reference>` paths need to be re-verified. Ideally there should be no references outside the current module for any given .ts. TODO: The embedded vs browser vs node (headless) builds should be tidied up for consistency so that it's obvious what depends on what. This is currently messiest in the predictive-text folder, where the output names diverge from the filenames and the various files are mixed in the same folder (as evidenced by the exclusions listed in each tsconfig.json). TODO: `npm install` should be removed from most build scripts and instead `npm ci` (#6196) should be run only once from the top-level folder for any given build. I've had eliminated side-effects from the `install` action for npm, which makes it easier to reason about state. TODO: verify_npm_setup and related functions can probably be eliminated. TODO: most of the build scripts should be largely eliminated for web. TODO: several ts projects use inconsistent output folders. TODO: it may be possible to generate a .d.ts for models/types so that we can use a consistent reference for those as well. TODO: build.sh, tsconfig.json should always be in the module's top-level folder, not in a subfolder such as src (e.g. see input-processor/src, keyboard-processor/src, web/source). TODO: resources/web-environment should be in common/web. TODO: other js node_modules imports should be wrapped like es6-shim. TODO: fix up the publish code for npm modules TODO: eliminate version numbers from package.json if possible? Whew, that's most of the stuff I noticed!
User Test ResultsTest specification and instructions
|
Shouldn't this be a command-line option on our build scripts instead? Or would we need to manually go out of our way to manually run Keep in mind, that's a not-insignificant amount of data each Maybe we even have the |
Are you suggesting that we'll be able to have the isolated component tests run from the main project's folder? I'd think we'd need to address the original idea of #5024 in order to do so successfully. Each package's folder has very useful unit tests, and I'd think we'd need their build scripts in order to keep 'em running at this time. |
|
When I cloned a new copy of the repo to examine this - so that it doesn't pollute my main working area - I got this error during the Web build: Obviously, not the hardest thing to resolve. But something of note. Next issue: It sounds as if the WebWorker-internal workspace is not being built. (It has its own ts-config.json.) Kinda need that for predictive text to work properly... and this means that Web isn't being fully compiled as of yet. My thoughts toward a possible resolution:
It's quite possible that this could be simplified; I just based this on my current level of understanding of the new "toys" you're "playing" with. I'm fairly certain this breakdown would work if what you're already doing works for es6-shim. Both issues above became evident rapidly because it's a clean workspace (fresh checkout). |
|
Thanks for the quick review @jahorton. Some quick notes:
|
This starts the reorg required for splitting the lm worker into its own module. Tests are still not working. But builds are going. Some decisions to be made around the final path for wrapping of the code. I think it may be possible to do something a whole lot more elegant by moving the wrapping process into the KMW build and leaving the LM Worker output as a pure tsc output -- given only KMW wants the wrapped worker anyway (tests aside). The actual wrapping code could still be in the lm-worker folder.
Starts to resolve the chaos of putting version numbers into every package.json file. Adds a new keyman-version package which mimics the version information in build-utils.sh. This will eventually replace resources/web-environment. Fixes the build for the lexical model compiler tools to remove refs to package.json version and use keyman-version instead. More fixes coming in a follow-up.
…up-unit-test-fixups
…e/web/merge-web-version-environment
…to chore/web/better-lm-worker-bundling
chore(web): restores previous test-setup patterns for Web unit tests 🎡
…nvironment chore(web): replaces web-environment with keyman-version 🎡
…ndling chore(web): lm-worker "test stub" code elimination 🎡
…developer chore(web): adapts keyman-version bundling in Developer products 🎡
Changes have been addressed. Will convert to "approve" once user tests are positive.
Workflow-wise, we don't merge until all three of the following are satisfied:
So please go ahead and approve if you are satisfied with the code changes; there is no need to wait for user tests to pass. (Note: if changes are required as a result of user testing, then we can dismiss reviews as required.) |
Test Results
|
|
Changes in this pull request will be available for download in Keyman version 16.0.12-alpha |
Fixes #6320.
Part 1 of moving from lerna to a simpler, maintained monorepo solution, using TypeScript Projects and NPM Workspaces.
There is more work to be done here. At this point, KeymanWeb builds unminified and runs without errors (locally at least!), but the built file is substantially different, mostly in include order.
Using TypeScript Projects, we move away from the need to run build scripts in various locations for almost all the Typescript modules.
npm installornpm ciis needed just once at the top level of the repo. The TypeScript project references mirror the npm references. A discussion may be worthwhile on how we link the two (or if we do) -- this seems to be an open question for TypeScript currently.On my machine, this reorg reduces the KeymanWeb build from several minutes to about 15 seconds! There may be more to do though.
TODO
<reference>paths need to be re-verified. Ideally there should be no references outside the current module for any given .ts. chore(web): bulk renderer tsc cleanup 🎡 #6616npm ciinstead ofnpm install(ref chore(common): move tonpm ciinstead ofnpm installfor builds #6196)For the build script and other cleanup phases (not part of this PR chain):
Note, remaining TODO items moved to #6268 (may be moved again from there).
Whew, that's most of the stuff I noticed!