Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate declaration files #9895

Closed
wants to merge 29 commits into from
Closed

Conversation

Rich-Harris
Copy link
Member

@Rich-Harris Rich-Harris commented May 10, 2023

Another run at #9883. We can't use .js files as the source of truth inside a package in node_modules, but we can use them as the source of truth for the .d.ts files. Combined with the declarationMap option, this makes 'go to definition' work as expected, and means that we don't have interfaces and documentation split between .d.ts and .js files.

This is WIP but I'm feeling fairly good about it. Downsides:

  • we need to re-run tsc if we change interfaces while developing (but we still get HMR etc for non-interface changes, which is most of them)
  • we need to put types that can't be expressed in JSDoc in .ts files, but in order to retain the benefits of shipping raw source code, we need to ensure that we only export types, not values, from those files. It might be worth having a build time check to guarantee that

Update

Ok, so this mostly works (though the logic for generating docs from types needs to be updated). Summary:

  • documentation and function interfaces etc live in source .js files
  • we run tsc to turn src/**/*.js into types/**/*.d.ts (and *.d.ts.map, so 'go to definition' goes to the source)
  • we copy over src/**/*.d.ts to types
  • we adjust all the import('types') stuff to use relative imports, otherwise types will be broken for people not using moduleResolution: 'bundler'
  • pkg.exports points at the source files, as before, but the . export additionally has a types condition which points at types/exports/types.d.ts (copied from src/exports/types.d.ts)
  • pkg.types and pkg.typesVersions are configured so that all this continues to work without moduleResolution: 'bundler'

Trade-offs:

  • VSCode will no longer autocomplete @sveltejs/kit/*, only @sveltejs/kit itself. This is rather annoying. I'm not sure if there's a piece missing that would fix this. We could generate the declare module declarations from the .d.ts files, I suppose, though that would kinda suck
  • This adds a prepublish step where there wasn't one before. I think this trade-off is worthwhile, but it deserves to be called out

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

  • It's really useful if your PR references an issue where it is discussed ahead of time. In many cases, features are absent for a reason. For large changes, please create an RFC: https://github.com/sveltejs/rfcs
  • This message body should clearly illustrate what problems it solves.
  • Ideally, include a test that fails without this PR but passes with it.

Tests

  • Run the tests with pnpm test and lint the project with pnpm lint and pnpm check

Changesets

  • If your PR makes a change that should be noted in one or more packages' changelogs, generate a changeset by running pnpm changeset and following the prompts. Changesets that add features should be minor and those that fix bugs should be patch. Please prefix changeset messages with feat:, fix:, or chore:.

@changeset-bot
Copy link

changeset-bot bot commented May 10, 2023

⚠️ No Changeset found

Latest commit: b56bbb5

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Copy link
Member

@dummdidumm dummdidumm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using the JSdoc output without further cleanup looks like this on hover:
image
which is not very pleasing an the @template is straight up confusing for someone not familiar with it. This function makes it nicer:

function adjust(input) {
	// Extract the import paths and types
	const import_regex = /import\(("|')(.+?)("|')\)\.(\w+)/g;
	let import_match;
	const import_map = new Map();
  
	while ((import_match = import_regex.exec(input)) !== null) {
		const imports = import_map.get(import_match[2]) || new Set();
		imports.add(import_match[4]);
		import_map.set(import_match[2], imports);
	}
  
	// Replace inline imports with their type names
	const transformed = input.replace(import_regex, "$4");
  
	// Remove/adjust @template, @param and @returns lines
	const lines = transformed.split("\n");
	const filtered_lines = lines.filter(line => {
		line = line.trim();
	  	return !line.startsWith("* @template") &&
			!((line.startsWith("* @returns") || line.startsWith("* @param")) && line.endsWith("}"));
	}).map(line => {
		line = line.trim();
		// Strip {...} from @param and @returns lines
		if (line.startsWith("* @param {") || line.startsWith("* @returns {")) {
			let openCount = 1;
			let closedCount = 0;
			let i = line.indexOf('{') + 1;
			for (; i < line.length; i++) {
				if (line[i] === "{") openCount++;
				if (line[i] === "}") closedCount++;
				if (openCount === closedCount) break;
			}
			line = line.slice(0, line.indexOf('{')) + line.slice(i + 1);
		}
		return line;
	});
  
	// Replace generic type names with their plain versions
	const renamed_generics = filtered_lines.map(line => {
	  return line.replace(/(\W|\s)([A-Z][\w\d$]*)_\d+(\W|\s)/g, "$1$2$3");
	});
  
	// Generate the import statement for the types used
	const import_statements = Array.from(import_map.entries())
	  .map(([path, types]) => `import { ${[...types].join(', ')} } from '${path}';`)
	  .join("\n");
  
	return [import_statements, ...renamed_generics].join("\n");
}

* return an error response without invoking `handleError`.
* Make sure you're not catching the thrown error, which would prevent SvelteKit from handling it.
* @param {number} status The [HTTP status code](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status#client_error_responses). Must be in the range 400-599.
* @param {{ message: string } extends App.Error ? App.Error | string | undefined : never} message An object that conforms to the App.Error type. If a string is passed, it will be used as the message property.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not the same as the original definition, you need to define the overload as in https://devblogs.microsoft.com/typescript/announcing-typescript-5-0-beta/#overload-support-in-jsdoc, for which we need to update the TS in the repo to 5.0

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added @overload

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gah, this causes the build to fail because suddenly it doesn't know what App is (even though it was fine in the original definition). any ideas?

to be honest I'm still unclear on why we need the overload

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah nvm figured it out, it was unrelated — i had removed .ts files from tsconfig.build.json

Copy link
Member

@dummdidumm dummdidumm May 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need the overload because the type definition is:

  • if you did not modify App.Error, in other words only { message: string } is required, then the second parameter is optional (because it is falling back to Error: ${status} in that case)
  • if you did modify App.Error, in other words more than just { message: string } is required, then you can't omit the second parameter and it is required

You can only define this with overloads

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gotcha, i missed the fact that one was optional and one wasn't. updated

* @param {Record<string, any> | undefined} [data]
* Create an `ActionFailure` object.
* @param {number} status The [HTTP status code](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status#client_error_responses). Must be in the range 400-599.
* @param {Record<string, any> | undefined} [data] Data associated with the failure (e.g. validation errors)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not the same function definition, you need to use @template to define the generic so it returns properly. It's probably also good explicitly annotate the return type (also on the other functions).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated, but i think i screwed it up somehow: #9895 (comment)

Comment on lines 96 to 100
* Create an `ActionFailure` object.
* @template {Record<string, any> | undefined} T
* @param {number} status The [HTTP status code](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status#client_error_responses). Must be in the range 400-599.
* @param {T} [data] Data associated with the failure (e.g. validation errors)
* @returns {ActionFailure<T>}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's something about this change that's causing one test to fail — the ActionResult here...

...has this type:

ActionResult<{
    status: number;
    data: {
        fail: string;
    } | undefined;
} | {
    success: boolean;
} | {
    id: number;
    username: string;
    profession: string;
} | {
    status: number;
    data: {
        reason: {
            error: {
                code: "VALIDATION_FAILED";
            };
        };
    } | undefined;
} | {
    ...;
}, never>

It should have this type:

ActionResult<{
    success: boolean;
} | {
    id: number;
    username: string;
    profession: string;
}, {
    fail: string;
} | {
    reason: {
        error: {
            code: "VALIDATION_FAILED";
        };
    };
}>

In other words, the ExcludeActionFailure and ExtractActionFailure types in the generated $types.d.ts are misfiring:

type ExcludeActionFailure<T> = T extends Kit.ActionFailure<any> ? never : T extends void ? never : T;
type ActionsSuccess<T extends Record<string, (...args: any) => any>> = { [Key in keyof T]: ExcludeActionFailure<Awaited<ReturnType<T[Key]>>>; }[keyof T];
type ExtractActionFailure<T> = T extends Kit.ActionFailure<infer X>	? X extends void ? never : X : never;
type ActionsFailure<T extends Record<string, (...args: any) => any>> = { [Key in keyof T]: Exclude<ExtractActionFailure<Awaited<ReturnType<T[Key]>>>, void>; }[keyof T];
type ActionsExport = typeof import('../../../../../../../../+page.server.js').actions
export type SubmitFunction = Kit.SubmitFunction<Expand<ActionsSuccess<ActionsExport>>, Expand<ActionsFailure<ActionsExport>>>

Instead of getting a SubmitFunction<Success, Failure> we're getting something like a SubmitFunction<Success & Failure, never>. I'm not sure why.

Copy link
Member

@dummdidumm dummdidumm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One way to solve the auto completion problem would be to not use typesVersions and instead create d.ts files which TypeScript will naturally pick up:

kit
  hooks.d.ts -> just reexports everything from types/../hooks/index.ts or wherever
  vite.d.ts -> same
  node/index.d.ts -> same
  node/polyfills.d.ts -> same

This pollutes the project a bit but I'm willing to take that hit for better DX.

In general my gripe with this change is that things are now split up. Interfaces are at X, methods at Y, it's all a bit scattered - maybe I'm just not used to it. This is the one thing TS makes better IMO, you can colocate this stuff better.

@@ -18,8 +16,9 @@ import {
RequestOptions,
RouteSegment,
UniqueInterface
} from './private.js';
} from './private';
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK all type imports need the .js ending or else moduleResolution: node16/nodenext will error. It's stupid, but it is what it is.

export class Server {
constructor(manifest: SSRManifest);
export interface Server {
new (manifest: SSRManifest): Server;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this change? Curious if these are equivalent.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah this is because at one point i was experimenting with having .ts files in src instead of .d.ts files, so that all the .d.ts files would be generated. (i think having .d.ts files in src is better, on reflection — we just need to copy them over to the types directory). you can't declare a class in a .ts file without actually implementing it

now that it's in a .d.ts again we could revert

@Rich-Harris
Copy link
Member Author

One way to solve the auto completion problem would be to not use typesVersions and instead create d.ts files which TypeScript will naturally pick up:

Unfortunately VSCode is liable to create broken imports like this...

import { thing } from '@scope/package/subpackage/index.js';

...when you want this:

import { thing } from '@scope/package/subpackage';

It's dependent on the user's config etc, so it's unreliable.

In general my gripe with this change is that things are now split up. Interfaces are at X, methods at Y, it's all a bit scattered - maybe I'm just not used to it. This is the one thing TS makes better IMO, you can colocate this stuff better.

TS definitely is better on this dimension, though I actually think this PR is better than what we currently have — all the types are in .d.ts files, all the values are in .js files, instead of having types and values in .d.ts files and having values in .js files. It feels like a cleaner separation to me.

Though it is a shame that we have to use declare module for all the $app stuff, and export functions and stuff from there. Would love to figure out a way to drive that from source (not least for 'go to definition'), but I haven't figured out what that could look like

@Rich-Harris
Copy link
Member Author

After playing around a bit more I've come to the conclusion that what we really want is to generate (from source, with sourcemaps) a single .d.ts that bundles all the types and uses declare module:

declare namespace App {
  export interface Error {...}
}

declare module '@sveltejs/kit' {
  export interface Adapter {...}
  export function error(status: number, body: App.Error): ...
  // etc
}

declare module '@sveltejs/kit/hooks' {
  import { Handle } from '@sveltejs/kit';
  export function sequence(...handlers: Handle[]): Handle;
}

declare module '$app/environment' {
  export const browser: boolean;
  // ...
}

// etc

//# sourceMappingURL=index.d.ts.map

The index.d.ts.map file would map back to the original source.

This is the most reliable way to get auto-imports working across all TypeScript configurations, and is the only way to make it work with modules like $app/environment, since they can't be expressed in a pkg.exports map. It's basically what we have now, except generated from source instead of handwritten, which would a) simplify the codebase and b) allow us to map back to the original source. Additionally, having everything in a single file ought to speed up startup (i.e. less time looking at a 'loading...' indicator while the TS server figures out where everything is).

Unfortunately, a tool for creating such a .d.ts bundle doesn't currently exist. There's a couple of tools that are related (api-extractor, rollup-plugin-dts, dts-bundle) but none of them are really suitable for this use case. I think such a tool should exist and I wouldn't mind having a go at building it, but I can't pretend it's a top priority.

@Rich-Harris
Copy link
Member Author

going to close this and maybe take another run at it later

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants