Doesn't reference pages in subdirectories

[samccann]

I tried running this against the Ansible documentation, which includes multiple subdirectories, and I only get results from the parent directory.

Generated sitemap

https://docs.ansible.com/latest/404.html
https://docs.ansible.com/latest/genindex.html
https://docs.ansible.com/latest/py-modindex.html
https://docs.ansible.com/latest/search.html
https://docs.ansible.com/latest/opensearch.html

Details

• sphinx version 1.8.5
• source - https://github.com/samccann/ansible/tree/sitemape/docs/docsite
• conf.py - https://github.com/samccann/ansible/blob/sitemape/docs/docsite/rst/conf.py
• website - https://docs.ansible.com/ansible/devel/index.html

[jdillard]

It looks like this is due to sphinx running in multiprocessor mode. When I remove -j 4 from the build command, it seems to include all the pages.

I think the fix is to figure out how to move the sitemap_links list into shared memory. I haven't figured out how to do that just yet, but it probably looks something like sphinx.ext.todo.

Once that is working, the following can be added to setup:

return {
    'parallel_read_safe': True
}

[samccann]

Tried that locally (removing -j ) and it does seem to have all the entries in there now.

[jdillard]

Great. It looks like it should be switching to a serial read, because it throws these warnings in the beginning:

WARNING: the sphinx_sitemap extension does not declare if it is safe for parallel reading, assuming it isn't - please ask the extension author to check and make it explicit
WARNING: doing serial read

But maybe it isn't switching properly and there is a bug somewhere preventing that. I'll keep looking, but it's not looking like a small change.

[samccann]

no worries. Thanks for digging into this. I at least have a way to create it when I need to.

[toshiakiasakura]

I was also stuck in this problem, when I developed my own OSS.
I used "nbsphinx" to generate a site. At first, I thought this was source of a problem, but I found this issue and I suspected that subdirectories might be a cause.

Also, I could not figure out the above discussion. How to remove -j 4, since I always type sphinx-build ./docs_src ./docs to create docs, there is no -j 4 for me... Also, if setting return { 'parallel_read_safe': True} solves this issue, I also can not figure out where to set it. I just tried to set the file instructed in sphinx.ext.todo, but the problem was not solved...

Details

• Source : https://github.com/toshiakiasakura/contextplt/tree/sitemap_bug
• conf.py : https://github.com/toshiakiasakura/contextplt/blob/sitemap_bug/docs_src/conf.py
• Site : https://toshiakiasakura.github.io/contextplt/