perf: minimize application preload payload

[cwaring]

Currently the blog index is loading over 5.2mb of JS which is blocking the site rendering for nearly 10s. This payload needs to be significantly reduced to meet performance standards and introduce async lazy loading for additional page meta.

related: #40 #30

[jessicaschilling]

@zebateira - per our conversation, are you able to work with @cwaring and @jdiogopeixoto to get this and #34 resolved? As we discussed, we might also want to look into establishing some sort of ongoing instrumentation for the blog to keep an eye on performance as the blog continues to expand over time. Thank you!

[jessicaschilling]

Note re instrumentation: @atopal recommends https://speedcurve.com/.

[jessicaschilling]

@zebateira and @jdiogopeixoto - I've added #84 as a larger issue for adding ongoing performance instrumentation to the blog (as well as other IPFS websites). In the meantime, the high-level metrics on WebPageTest don't look as bad as I'd feared compared to the existing blog metrics, but are you able to look into these in closer detail to identify other areas of optimization? We can open a separate issue if that's easier. Thank you!

cc @atopal, per his request.

[zebateira]

@zebateira and @jdiogopeixoto - I've added #84 as a larger issue for adding ongoing performance instrumentation to the blog (as well as other IPFS websites). In the meantime, the high-level metrics on WebPageTest don't look as bad as I'd feared compared to the existing blog metrics, but are you able to look into these in closer detail to identify other areas of optimization? We can open a separate issue if that's easier. Thank you!

This is great 🙏
It looks like this payload issue is the main one showing up in the metrics, so I'll go over this issue to get a fix in and then we can re-evaluate other performance issues.

[zebateira]

Progress report 1

I'll continue next week, but I wanted to leave some of the progress I have so far: https://www.notion.so/protocollabs/Performance-issue-on-IPFS-Blog-74fae8476b3e413abdde7704a9170a02

The main issue that I have found is that the whole entire list of blog posts is downloaded on the homepage (but not in the blog post page) which includes the blog post parsed markdown data as well. You can read the details in the Notion page above and to understand how I reached that conclusion.
I'm still looking at what is causing this to happen.

[cwaring]

👋🏻 hi folks, so I've investigated this a little further over the weekend and unfortunately it is related to a core component of how Vuepress 1.x bundles the application data. Essentially the frontmatter for every page is parsed and loaded into memory, designed this way to power the default theme search box, but as we're not using that functionality here it causing wasteful clientside overhead.

Related issues:

vuejs/vuepress#2689
vuepress/core#8

Good news this is fixed in Vuepress 2.x, bad news this is still in alpha and some essential plugins that we depend on (such as vuepress-plugin-blog have not been upgraded yet.

Next options, as I currently see them:

1. Accept that Vuepress 1.x is not a suitable solution for a site of this size and focus efforts on a 2.x migration:
   Risky if the timeline is short given the missing plugins and alpha build status.

2. Attempt to post process the application bundle to remove the unnecessary payload:
   There are plenty of ways to hook into the build process, this will fix the client-side issues but add additional processing overhead to the webpack bundle (which is already slow)

3. Migrate to another framework for the blog such as nuxt or gridsome:
   This will require significantly more custom development but we can control the pipeline and output precisely.

[cwaring]

One additional note, when I originally evaluated these bundlers Vuepress was the only one I that I could get to work with relative path urls suitable for meeting the requirement of hosting on IPFS (using vuepress-plugin-ipfs). At the time we didn't have an origin domain proxy in IPFS ({{CID}}.ipfs.dweb.link) so this was a critical implementation, if this is no longer a hard requirement we have other framework options at our disposal.

[jessicaschilling]

We have discussed using nuxt on ipfs.io, as well as in separate discussion regarding the ecosystem directory being built by @orvn.

Really reluctant to blow our timeline out to this degree but how lengthy would migrating this blog to nuxt be? @cwaring @jdiogopeixoto @zebateira, please weigh in here?

[zebateira]

@jessicaschilling you've said before that considering our current deadline goal for go-live and the fact that we are already in a position that the current new ipfs blog is already an improvement to the old one even considering this performance issue.

I agree with going live with this issue, and then we can fix this either by moving to nuxt.js or upgrading to vuepress 3.
We would most likely not have enough time to migrate to nuxt.js and make it in time to publish by the end of the month.

[jessicaschilling]

Responding to @cwaring's analysis in #41 (comment):

Next options, as I currently see them:

1. Accept that Vuepress 1.x is not a suitable solution for a site of this size and focus efforts on a 2.x migration:
   Risky if the timeline is short given the missing plugins and alpha build status.
2. Attempt to post process the application bundle to remove the unnecessary payload:
   There are plenty of ways to hook into the build process, this will fix the client-side issues but add additional processing overhead to the webpack bundle (which is already slow)
3. Migrate to another framework for the blog such as nuxt or gridsome:
   This will require significantly more custom development but we can control the pipeline and output precisely.

Suggest that we rule out option 1 in favor of both of the following:

• Pursue option 2 in the near-term to improve payload as much as possible in advance of an end-March launch. As @cwaring notes, this does add time to the webpack bundle, which already isn't great, but doesn't seem like a dealbreaker in the medium term, and impacts us, not the user.
• Pursue option 3 as nuxt replatform later in the year after replatforming the MAIN ipfs.io to nuxt: Per where the discussion seems to be going in [DISCUSSION] Site replatform alignment ipfs-inactive/website#412, as well as recommendation by Agency Undone to use nuxt for the ecosystem directory in https://github.com/ipfs/website/issues/410, it's tentatively looking like nuxt may be the way forward. If we can agree here that nuxt is also an appropriate choice for the blog in the long term, this may seal the deal for a platform decision.

REQUEST FOR THE REST OF THE TEAM
@cwaring @zebateira @jdiogopeixoto -- What are your thoughts on proceeding as suggested? I realize my suggestion on path forward is based more from a project management perspective (particularly since we have three items in flight right now that need platform alignment), not a purely technical one. I'm trying to balance both angles while also not wanting to block any work.

(cc @orvn @D3AS8A for context re ecosystem directory)

[cwaring]

This sounds like a suitable approach. Vuepress is still going to be a useful tool in our arsenal for docs and simple sites, plus the benefits included in the upcoming v2.x will solve the main problem in this issue and speed up the entire build pipeline overall, so I don't see us moving away from it completely.

Nuxt is a logical next step for more power and flexibility, however as @zebateira has mentioned we need to build a custom boilerplate that would make sense for us and commit more resources to maintaining our web properties. This is going to take time and effort to get right.

I think the happy place is if we commit the effort to support both at PL, and design a common UI kit that can be shared across all Vue applications. This will be help our teams be productive and provide more value to the community looking to deploy IPFS friendly web-apps.

p.s:
All this could happen without blocking any development for ipfs.io, as that is a 10 page site max it should be a breeze to start in Vuepress and port to another if we get the interface layer correct.

[cwaring]

Practically speaking we could also shave off some build time by removing any plugins that are not required by the blog.

Please be aware that the backend slowness is a webpack v4 problem (which is doing all the bundling work under the hood) rather than a vuepress issue, and considering that Nuxt v2 is also running webpack v4 we may not gain a great deal in compile time by migrating across, we may have to wait for Nuxt v3 and incremental builds to ship. In that regard vuepress-next and vite are already ahead with solving some of these core problems.

[zebateira]

@cwaring @zebateira @jdiogopeixoto -- What are your thoughts on proceeding as suggested? I realize my suggestion on path forward is based more from a project management perspective (particularly since we have three items in flight right now that need platform alignment), not a purely technical one. I'm trying to balance both angles while also not wanting to block any work.

Agreed 👍
It's compromise and not at all a problem IMO.

Nuxt is a logical next step for more power and flexibility, however as @zebateira has mentioned we need to build a custom boilerplate that would make sense for us and commit more resources to maintaining our web properties. This is going to take time and effort to get right.

I wouldn't add this as a must to the task of starting ipfs.io replatform - it would help but we would have a lot more lessons to learn from the first usage of Nuxt.js and incorporate that into the boilerplate later.

[zebateira]

2. Attempt to post process the application bundle to remove the unnecessary payload:
   There are plenty of ways to hook into the build process, this will fix the client-side issues but add additional processing overhead to the webpack bundle (which is already slow)

Regarding this @cwaring what do you suggest doing exactly? Using chainWebpack?

[jdiogopeixoto]

Sorry for being awfully quiet during this discussion. As I am relatively new to the Vue ecosystem I didn't feel like I had any valuable input.

I did came across those issues that @cwaring mentioned about scalability when I was trying to find a solution for this, I should have mentioned them earlier.

There's something that I tried that might be worth mentioning here. I tried to use the official blog theme (vuepress-theme-blog) with all our blog posts. After building it the biggest file I got was about 800 KB with the next one being about 70KB - a huge difference compared to our files.

What I noticed is that in their main JS file all the content from the blog posts is truncated and in our main JS file we have all the content (that we don't really use). So before doing any webpack improvements I would suggest we try to do the same and see by how much it reduces our bundle size.

[cwaring]

Nice find @jdiogopeixoto, yes I suspect there is a lot more that can be done starting with a simple cleanup before messing with webpack - considering this is a temporary situation for a few months that is a fair compromise to get this project over the line.

✨ Also bonus tip✨ : the next vuepress is using esbuild which is like a rocket compared to webpack. Excited for this to land so we can upgrade all our apps!

[jdiogopeixoto]

I've been messing around with the project trying to find the issue. I think I managed to solve it by removing some files and changing some things. I opened a PR (#91) with the changes.

@cwaring @zebateira can you take a look at it?

[zebateira]

@jdiogopeixoto looking good! 🎉