Long term plans for Documentation

[dstufft]

Right now PyPI supports the ability for people to upload zip files containing raw HTML that we'll then host at https://pythonhosted.org/{projectname}/. Ideally I'd like to stop being in the documentation hosting business and instead direct people to use ReadTheDocs. This was proposed before on distutils-sig however it was ultimately shot down because some people don't use Sphinx in their documentation toolchain and they want/need to be able to upload raw html.

After talking with @ericholscher this is a use case that ReadTheDocs wants to support, so I believe once RTD does, we can work on moving people over to using that instead. This issue will be about capturing what work needs done to enable such a transition.

In addition it would be great to have better integration between RTD and PyPI. Currently having your documentation hosted on PyPI gives a somewhat better link to the documentation than having that documentation in RTD does. We should prefer integrations that bias users towards using RTD and try to figure out what an ideal integration looks like. It's possible that this could be used as a model for additional integrations, both for Warehouse integrating with other services (code quality? i18n?) and for other package repositories wanting to integrate with RTD.

[ericholscher]

+1

Not sure what this would look like. I've previously thought about doing something like:

project.pypi.readthedocs.org -- or perhaps we could continue to use a CNAME for this and keep it top-level, so RTD would support project.pypidocs.org or we could continue to use the existing namespace at packages.python.org/<project>.

I'd love to work to figure out a way to auto-build docs for packages that are uploaded. It would be neat to have this built into the python toolchain, similar to python setup.py test have a python setup.py doc or something. Though to make my world make sense, we'd likely just do a Sphinx build on all markdown & rst files we found, along with auto-generating an API reference from the python files.

[domenkozar]

+1

[dstufft]

So, as I posted on distutils-sig here's the plan:

1. Implement the ability to delete documentation.
2. Implement the ability to add a (simple) redirect where we would essentially just send /<project>/(.*) to $REDIRECT_BASE/$1.
3. Implement the ability to point the documentation URL to something that isn't pythonhosted.org
4. Send an email out to all projects that are currently utilizing the hosted documentation telling that it is going away, and give them links to RTD and Github Pages and whatever BitBucket calls their service.
5. Disable documentation Uploads to PyPI with an error message that tells people the service has been discontinued.

In addition to the above steps, we'll maintain any documentaton that doesn't get deleted (and the above redirects) indefinitely. Serving static read only documentation (other than deletes) is something that we can do without much trouble or cost.

Most of that work will belong as part of PyPI legacy, but Warehouse will need:

• Descriptive error on documentation upload.
• Ability to delete existing documentation.
• Ability to set up a redirect from pythonhosted.org to the new location.
• Ability to define a specific documentation URL for a project that isn't pythonhosted.org.

[ericholscher]

+1 on this plan. GitHub Pages is great for arbitrary HTML uploads. They
even support CNAME's.

I think having a solid RTD & Pypi integration would be really cool:

• Autogenerate "javadoc" style API references from the source code that is
  published in the package.
• Build Sphinx documentation from the RST and Markdown files that are in
  the release of the package.
  • Optionally support a readthedocs.yml file inside the project to
    configure doc building
• Link these two together on our side inside of a TOCTree.

This would allow for a really tight integration with Pypi. We could then
serve these packages at either:

• .pypi.readthedocs.org
• .package.python.org

Or at a CNAME configurable in the YAML file.

On Wed, Jun 3, 2015 at 6:27 AM, Donald Stufft notifications@github.com
wrote:

So, as I posted on distutils-sig
https://mail.python.org/pipermail/distutils-sig/2015-May/026381.html
here's the plan:

1. Implement the ability to delete documentation.
2. Implement the ability to add a (simple) redirect where we would
   essentially just send //(.*) to $REDIRECT_BASE/$1.
3. Implement the ability to point the documentation URL to something
   that isn't pythonhosted.org
4. Send an email out to all projects that are currently utilizing the
   hosted documentation telling that it is going away, and give them links to
   RTD and Github Pages and whatever BitBucket calls their service.
5. Disable documentation Uploads to PyPI with an error message that
   tells people the service has been discontinued.

In addition to the above steps, we'll maintain any documentaton that
doesn't get deleted (and the above redirects) indefinitely. Serving static
read only documentation (other than deletes) is something that we can do
without much trouble or cost.

Most of that work will belong as part of PyPI legacy, but Warehouse will
need:

• Descriptive error on documentation upload.
• Ability to delete existing documentation.
• Ability to set up a redirect from pythonhosted.org to the new
  location.
• Ability to define a specific documentation URL for a project that
  isn't pythonhosted.org.

—
Reply to this email directly or view it on GitHub
#509 (comment).

Eric Holscher
Maker of the internet residing in Portland, Oregon
http://ericholscher.com

[dstufft]

I've added the above steps as their own issues in #582 and #583 and I've already merged #581. Since there isn't a real actionable item in this issue and I don't think we need an issue to remind us we want to integrate more with RTD I'm going to go ahead and close this.

[serge-sans-paille]

Converning the

Ability to set up a redirect from pythonhosted.org to the new location.

item, I failed to find how to do that...

[brainwane]

@serge-sans-paille We are following up on that question in #582. Sorry for the confusion.