Add experimental content layer flag #11652
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2184,6 +2184,259 @@ export interface AstroUserConfig { | |
| * For a complete overview, and to give feedback on this experimental API, see the [Server Islands RFC](https://github.com/withastro/roadmap/pull/963). | ||
| */ | ||
| serverIslands?: boolean; | ||
|
|
||
| /** | ||
| * @docs | ||
| * @name experimental.contentLayer | ||
| * @type {boolean} | ||
| * @default `false` | ||
| * @version 4.16.0 | ||
| * @description | ||
| * | ||
| * The Content Layer API is a new way to handle content and data in Astro. It builds upon [content collections](https://docs.astro.build/en/guides/content-collections/), taking them beyond local files in `src/content` and allowing you to fetch content from anywhere, including remote APIs, or files anywhere in your project. As well as being more powerful, the Content Layer is designed to be more performant, helping sites scale to thousands of pages. Data is cached between builds and updated incrementally. Markdown parsing is also 5-10x faster, with similar scale reductions in memory. While the feature is experimental and subject to breaking changes, we invite you to try it today and let us know how it works for you. | ||
|
There was a problem hiding this comment. Docs comment re: terminology (with full understanding this can change in the experimental stage easily enough!) Sorry for the length! I see in the RFC we still use "collection" and "entry". As I'm going through the text, it feels the most natural to me (outsider, thinking of explaining this feature) to refer to "content layer" in e.g. the following ways
"content layer" doesn't feel like it can replace "content collections" one-to-one. You don't have "different layers." You do have "a content layer" but that's not instead of having collections, and that's not the main part of this that the user interacts with: they make collections that have entries. I don't think "layer collections" feels very ergonomic, but that's really the closest analogy. 😅 So again, we don't have to have this all nailed down, but as I'm reviewing this I'll be suggesting what I think feels both clear and helpful to someone using this new API to power their collections! "The layer" may be the underlying interesting tech, but I really think "collection" is going to still be the main word that resonates with users the most. Once the layer is powering everything, no one is going to be thinking of a "content layer loader" when they write their collection schema, it's a loader they define in their collection, using There was a problem hiding this comment. Yeah, I think the right summary is that these new content collections are powered by the content layer API There was a problem hiding this comment. @sarah11918 I've made some updates to the terminology. See if that's better. |
||
| * | ||
| * #### Enabling content layer | ||
| * | ||
| * To enable, add the `contentLayer` flag to the `experimental` object in your Astro config: | ||
| * | ||
| * ```js | ||
| * { | ||
| * experimental: { | ||
| * contentLayer: true, | ||
| * } | ||
| * } | ||
| * ``` | ||
| * | ||
| * #### Using the content layer | ||
| * | ||
| * :::tip | ||
| * The content layer APIs are based on content collections, so you can learn more about them in the [content collection docs](https://docs.astro.build/en/guides/content-collections/. Any differences are highlighted below. | ||
| * | ||
| * ::: | ||
| * | ||
| * To use content layer, create a collection in `src/content/config.ts` with a `loader` property. For local files where there is one entry per file, use the `glob()` loader. You can put your content files anywhere, but *not* in `src/content` because these would be handled by the current content collections instead. In this example the files are in `src/data`. | ||
ascorbic marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| * | ||
| * ```ts | ||
| * import { defineCollection, z } from 'astro:content'; | ||
| * import { glob } from 'astro/loaders'; | ||
| * | ||
| * const blog = defineCollection({ | ||
| * // By default the ID is a slug, generated from the path of the file relative to `base` | ||
| * loader: glob({ pattern: "**\/*.md", base: "./src/data/blog" }), | ||
| * schema: z.object({ | ||
| * title: z.string(), | ||
| * description: z.string(), | ||
| * pubDate: z.coerce.date(), | ||
| * updatedDate: z.coerce.date().optional(), | ||
| * }), | ||
| * }); | ||
| * | ||
| * export const collections = { blog }; | ||
| * ``` | ||
| * | ||
| * You can load multiple entries from a single JSON file using the `file()` loader. In this case the data must either be an array of objects, which each contain an `id` property, or an object where each key is the ID. | ||
| * | ||
| * **Array syntax:** | ||
| * | ||
| * ```json | ||
| * [ | ||
| * { | ||
| * "id": "labrador-retriever", | ||
| * "breed": "Labrador Retriever", | ||
| * "size": "Large", | ||
| * "origin": "Canada", | ||
| * "lifespan": "10-12 years", | ||
| * "temperament": [ | ||
| * "Friendly", | ||
| * "Active", | ||
| * "Outgoing" | ||
| * ] | ||
| * }, | ||
| * { | ||
| * "id": "german-shepherd", | ||
| * "breed": "German Shepherd", | ||
| * "size": "Large", | ||
| * "origin": "Germany", | ||
| * "lifespan": "9-13 years", | ||
| * "temperament": [ | ||
| * "Loyal", | ||
| * "Intelligent", | ||
| * "Confident" | ||
| * ] | ||
| * } | ||
| * ] | ||
| * ``` | ||
| * | ||
| * **Object syntax:** | ||
| * | ||
| * ```json | ||
| * { | ||
| * "labrador-retriever": { | ||
| * "breed": "Labrador Retriever", | ||
| * "size": "Large", | ||
| * "origin": "Canada", | ||
| * "lifespan": "10-12 years", | ||
| * "temperament": [ | ||
| * "Friendly", | ||
| * "Active", | ||
| * "Outgoing" | ||
| * ] | ||
| * }, | ||
| * "german-shepherd": { | ||
| * "breed": "German Shepherd", | ||
| * "size": "Large", | ||
| * "origin": "Germany", | ||
| * "lifespan": "9-13 years", | ||
| * "temperament": [ | ||
| * "Loyal", | ||
| * "Intelligent", | ||
| * "Confident" | ||
| * ] | ||
| * } | ||
| * } | ||
| * ``` | ||
| * | ||
| * The collection is then defined using the `file()` loader: | ||
| * | ||
| * ```ts | ||
| * import { defineCollection, z } from 'astro:content'; | ||
| * import { file } from 'astro/loaders'; | ||
| * | ||
| * const dogs = defineCollection({ | ||
| * loader: file('src/data/dogs.json'), | ||
| * schema: z.object({ | ||
| * id: z.string(), | ||
| * breed: z.string(), | ||
| * size: z.string(), | ||
| * origin: z.string(), | ||
| * lifespan: z.string(), | ||
| * temperament: z.array(z.string()), | ||
| * }), | ||
| * }); | ||
| * | ||
| * export const collections = { dogs }; | ||
| * ``` | ||
| * | ||
| * The collection can be queried in the same way as content collections: | ||
| * | ||
| * ```ts | ||
| * import { getCollection, getEntry } from 'astro:content'; | ||
| * | ||
| * // Get all entries from a collection. | ||
| * // Requires the name of the collection as an argument. | ||
| * const allBlogPosts = await getCollection('blog'); | ||
| * | ||
| * // Get a single entry from a collection. | ||
| * // Requires the name of the collection and ID | ||
| * const labradorData = await getEntry('dogs', 'labrador-retriever'); | ||
| * ``` | ||
| * | ||
| * #### Rendering content | ||
| * | ||
| * Entries generated from markdown or MDX can be rendered directly to a page using the `render()` function. | ||
| * | ||
| * :::caution | ||
| * The syntax for rendering content layer items is different from current content collections syntax. | ||
| * ::: | ||
| * | ||
| * ```astro | ||
| * --- | ||
| * import { getEntry, render } from 'astro:content'; | ||
| * | ||
| * const post = await getEntry('blog', Astro.params.slug); | ||
| * | ||
| * const { Content, headings } = await render(entry); | ||
| * --- | ||
| * | ||
| * <Content /> | ||
| * ``` | ||
| * | ||
| * #### Creating a content layer loader | ||
| * | ||
| * Content layer loaders aren't restricted to just loading local files. You can also use loaders to load or generate content from anywhere. The simplest type of loader is an async function that returns an array of objects, each of which has an `id`: | ||
| * | ||
| * ```ts | ||
| * const countries = defineCollection({ | ||
| * loader: async () => { | ||
| * const response = await fetch("https://restcountries.com/v3.1/all"); | ||
| * const data = await response.json(); | ||
| * // Must return an array of entries with an id property, or an object with IDs as keys and entries as values | ||
| * return data.map((country) => ({ | ||
| * id: country.cca3, | ||
| * ...country, | ||
| * })); | ||
| * }, | ||
| * // optionally add a schema | ||
| * // schema: z.object... | ||
| * }); | ||
| * | ||
| * export const collections = { countries }; | ||
| * ``` | ||
| * | ||
| * For more advanced loading logic, you can define an object loader. This allows incremental updates and conditional loading, and gives full access to the data store. See the API in [the draft RFC](https://github.com/withastro/roadmap/blob/content-layer/proposals/content-layer.md#loaders). | ||
| * | ||
| * ### Migrating a content collection to content layer | ||
| * | ||
| * If you have used [content collections](https://docs.astro.build/en/guides/content-collections/), content layer will be very familiar as most of the APIs are the same. You can convert an existing content collection to content layer if it uses markdown, MDX or JSON, with these steps: | ||
| * | ||
| * 1. **Move the collection folder out of `src/content`.** This is so it won't be handled as a current content collection. This example assumes the content has been moved to `src/data`. The `config.ts` file must remain in `src/content`. | ||
| * 2. **Edit the collection definition**. The collection should not have `type` set, and needs a `loader` defined. | ||
| * | ||
| * ```diff | ||
| * import { defineCollection, z } from 'astro:content'; | ||
| * + import { glob } from 'astro/loaders'; | ||
| * | ||
| * const blog = defineCollection({ | ||
| * // For content layer you do not define a `type` | ||
| * - type: 'content', | ||
| * + loader: glob({ pattern: "**\/*.md", base: "./src/data/blog" }), | ||
| * schema: z.object({ | ||
| * title: z.string(), | ||
| * description: z.string(), | ||
| * pubDate: z.coerce.date(), | ||
| * updatedDate: z.coerce.date().optional(), | ||
| * }), | ||
| * }); | ||
| * ``` | ||
| * | ||
| * 3. **Change references from `slug` to `id`**. Content layer collections do not have a `slug` field. You should use `id` instead, which has the same syntax. | ||
| * | ||
| * ```diff | ||
| * --- | ||
| * export async function getStaticPaths() { | ||
| * const posts = await getCollection('blog'); | ||
| * return posts.map((post) => ({ | ||
| * - params: { slug: post.slug }, | ||
| * + params: { slug: post.id }, | ||
| * props: post, | ||
| * })); | ||
| * } | ||
| * --- | ||
| * ``` | ||
| * | ||
| * 4. **Switch to the new `render()` function**. Entries no longer have a `render()` method, as they are now serializable plain objects. Instead, import the `render()` function from `astro:content`. | ||
| * | ||
| * ```diff | ||
| * --- | ||
| * - import { getEntry } from 'astro:content'; | ||
| * + import { getEntry, render } from 'astro:content'; | ||
| * | ||
| * const post = await getEntry('blog', params.slug); | ||
| * | ||
| * - const { Content, headings } = await post.render(); | ||
| * + const { Content, headings } = await render(post); | ||
| * --- | ||
| * | ||
| * <Content /> | ||
| * ``` | ||
| * | ||
| * The `getEntryBySlug` and `getDataEntryByID` functions are deprecated and cannot be used with content layer collections. Instead, use `getEntry`, which is a drop-in replacement for both. | ||
| * | ||
| * #### Learn more | ||
| * | ||
| * To see the full API look at [the RFC](https://github.com/withastro/roadmap/blob/content-layer/proposals/content-layer.md) and [share your feedback on the feature and API](https://github.com/withastro/roadmap/pull/982). | ||
| */ | ||
| contentLayer?: boolean; | ||
| }; | ||
| } | ||
|
|
||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, to make things simpler, I have mocked up something in a branch of docs itself with an idea of what I'm thinking: https://content-layer-draft--astro-docs-2.netlify.app/en/reference/configuration-reference/#experimentalcontentlayer
I did take take a little from the RFC and mix it in here. I mostly tried to laser-focus things a bit because it's already quite long, and for experimental docs, we can be a bit more minimal and let the RFC do some of the work.
Putting on my "seeing this for the first time, want to play with it" hat, I feel like it's the
loaderstuff that people need to get is fundamentally different, and most important, in all this. So I spent a bit more time there, and didn't spend as much time on the querying and rendering (even combined them into one section!) because they work so much like content collections do already.So see what you think about something that kind of takes this form! When we have a good shape, then we can squish it in here to this file!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's great. Is it on a branch somewhere? Also do you have any suggestions for a workflow for working on this? It's really hard editing inside the JSDoc!