PROBLEM STATEMENT: Find a way to have rusEFI documentation indexed

[rusefillc]

https://github.com/nelsonjchen/github-wiki-see-rs

need a box where we can drop a docker and park rusefi subdomain

See also isaacs/github#1683
See also github/docs#4115

[rusefillc]

@chuckwagoncomputing would you like root access to

[rusefillc]

https://github.com/djacquel/WikiToJekyll is an alternative

[chuckwagoncomputing]

@rusefillc I think we should do the Github Pages option. If you're on board with that, you should create a repo and set up a .rusefi.com subdomain to point to it.

https://docs.github.com/en/pages/configuring-a-custom-domain-for-your-github-pages-site/managing-a-custom-domain-for-your-github-pages-site

[rusefillc]

https://github.com/rusefi/rusefi.github.io created
GH is aware of custom domain now https://github.com/rusefi/rusefi.github.io/blob/main/CNAME

let's see
http://rusefi.github.io
https://rusefi.github.io
http://wiki.rusefi.com
https://wiki.rusefi.com

[rusefillc]

Now I am confused why does http://rusefi.github.io redirect me to http://wiki.rusefi.com?

[rusefillc]

[rusefillc]

certificate drama next?

[chuckwagoncomputing]

https://docs.github.com/en/pages/configuring-a-custom-domain-for-your-github-pages-site/troubleshooting-custom-domains-and-github-pages#https-errors

[rusefillc]

is TL,DR to wait an hour?
It has been a few and still

[chuckwagoncomputing]

Try removing and re-adding the custom domain in the repo settings.

[rusefillc]

will give it a couple of hours and re-add

[rusefillc]

IT HAS WORKED?!

[rusefillc]

[rusefillc]

Need help with all this :(

https://docs.github.com/en/pages/setting-up-a-github-pages-site-with-jekyll
And the plugin and config from https://github.com/djacquel/WikiToJekyll

[chuckwagoncomputing]

So it turns out that Github Pages only supports a subset of Jekyll plugins, so the WikiToJekyll plugin doesn't work. We can generate it with a Github Action though.

[chuckwagoncomputing]

So basically, at this point there's no purpose to using Github Pages. In fact we'll be ahead with the configurability of our own server.
New Plan: Use a Github Action to generate static site files and FTP them to the server.

As to which static site generator to use, I'm not sure yet. Either I don't understand Jekyll infrastructure, or it's deeply crippled for non-github-pages use.

[chuckwagoncomputing]

Status update:
My next idea was Readthedocs. Readthedocs has a "Github Import" feature, but it doesn't seem to know how to handle markdown properly.
So I decided to try Sphinx, which is what Readthedocs runs on. There is a markdown parser extension for it. This works reasonably well, but has a few issues I haven't figured out how to handle.

1. If you add a file to the table of contents, all of the headers (lines starting with # in markdown) in that file are listed. For example, the Home page appears in the index as 4 items:
   Click on the image below for the latest builds and downloads!
   Features
   Get Started!
   rusEFI in action!
   The Sphinx index system has a few options that I thought might fix this, but it seems like this is how things are getting passed from the parser, and it doesn't work.

2. Files that don't have a header aren't listed in the index. Github gets its titles from the .md file name, but this doesn't seem possible with Sphinx. However, adding headers to files would be an improvement in Github as well, so I don't see this as a big problem. If we need to modify the markdown parser to fix 1 though, we could probably fix this too.

3. I noticed that with sphinx-rtd-theme, table cells don't wrap, and on the Hardware table, you have to scroll horizontally to see more than a few columns, at least on my 1366x768 laptop screen. This is a downgrade from the Github wiki.

[chuckwagoncomputing]

Now I have tried using Pandocs to convert all the .md files to .rst first. This fixes problem 1, but breaks tables in a new and interesting way. Pandoc makes large cells multiple lines high, and Sphinx doesn't handle that properly.

[chuckwagoncomputing]

Now I have tried Gollum, which is what Github Wikis use behind the scenes. It has problems too.

1. Tables don't work, and I don't know why.
2. Links to pages in a directory don't work. Gollum docs say you can use --lenient-tag-lookup to make this work, but it doesn't.

Besides these, it's fairly nice.

[chuckwagoncomputing]

I have found MkDocs and it is beautiful.

[chuckwagoncomputing]

We still need to add "title" headers, but otherwise I'm happy with this. Demo at https://rusefi.chuckwagoncomputing.com/wiki/Home/

[chuckwagoncomputing]

OK, so I'm happy with how things have worked out with MkDocs, and am ready to PR it. I have one question though. Do we want to keep the Github Wiki? Both are compatible with the current state of the wiki, but if we take advantage of things MkDocs can do (Youtube embeds!) then the Github Wiki will be broken.
My vote is to drop it like a hot potato. :)

[rusefillc]

I vote for one wiki whichever is best

[chuckwagoncomputing]

Noting this here so you don't forget:
The main page of the wiki is https://rusefi.com/docs/wiki/Home/
Whatever URL you want to use will need to be redirected to that.
e.g.
https://rusefi.com/docs/wiki
https://rusefi.com/wiki
https://wiki.rusefi.com
Or as many as you like, though we should probably pick one as "official".

[rusefillc]

definitely not https://rusefi.com/docs/wiki :) I will add/fix secrets once we are closer to production

[rusefillc]

Depends on #144 legacy wiki

[rusefillc]

Depends on #145 home redirect

[rusefillc]

Depends on #146 self-links

[rusefillc]

Depends on #149

[nelsonjchen]

😅 , my terrible web app hack wasn't meant to be an instance'd thing (https://github-wiki-see.page/). I'm glad you guys are sorting that out though!

[chuckwagoncomputing]

I saw another option today that may be worth considering: Docusaurus.
facebook/docusaurus#2725

[rusefillc]

As of today https://github.com/robots.txt does not mention wiki. it looks like this issue is no longer an issue.

We now need to clean-up wiki1 and consider hiding rusefi_documentation

[rusefillc]

community/community#4992 :( anyway let me make progress

[rusefillc]

@chuckwagoncomputing do you still have the scripts which made https://rusefi.com/docs/wiki/Home/ happen? where can I see those?

i am happy to handle #148

[chuckwagoncomputing]

Here's my branch where I was testing it. I'll get it up to date and ready to PR when I have time.
https://github.com/chuckwagoncomputing/rusefi_documentation/tree/external-wiki-2

Also, I want to punch someone.

[chuckwagoncomputing]

Current status is that Google indexing of Wiki3 still sucks, but Bing and DuckDuckGo indexing is great.

[mi-hol]

Google indexing of Wiki3 still sucks

@chuckwagoncomputing I seem to remember a hint in SiteChecker about "duplicated content" and a note to configure a corresponding hint for Google. Should I dig for this info?

[chuckwagoncomputing]

I made a sitechecker account, and these are the issues listed under Indexability:

I'm not sure how big of issues these actually are. From my research, I don't think that not having a robots.txt is an issue.
Other tags in H1 and H2 are because of how MkDocs generates the header hotlinks. That could be turned off, but it's nice to have. If turned off, hotlinks would still work, they just wouldn't be discoverable.

[chuckwagoncomputing]

Here is the Duplicate Content section