Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOC: parallelize the number of Sphinx jobs by default #39364

Merged
merged 1 commit into from
Feb 17, 2021

Conversation

afeld
Copy link
Member

@afeld afeld commented Jan 24, 2021

I'm assuming most people building docs would want that to happen as quickly as possible, so this has Sphinx use the maximum parallelism possible by default. Corresponding Sphinx documentation.

  • closes #xxxx
  • tests added / passed
  • Ensure all linting tests pass, see here for how to run them
  • whatsnew entry

@afeld afeld added the Docs label Jan 24, 2021
@@ -39,7 +39,7 @@ class DocBuilder:

def __init__(
self,
num_jobs=0,
num_jobs="auto",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is num_jobs just a pandas thing? We use the -j flag for building the Cython / C extensions so wouldn't be opposed to just using that here for consistency as well, since it matches the Sphinx docs

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

num_jobs was there before; I didn't mess with the name. I like that num_jobs is more descriptive, but certainly understand the argument for consistent…arguments 😉

@jreback
Copy link
Contributor

jreback commented Feb 11, 2021

does this help?

@afeld
Copy link
Member Author

afeld commented Feb 11, 2021

Running on Codespaces, before and after:

$ git checkout upstream/master && python make.py clean && time python make.py --no-api
real    5m28.607s
user    5m58.308s
sys     1m50.928s
$ git checkout parallel-default && python make.py clean && time python make.py --no-api
real    3m25.720s
user    6m47.912s
sys     1m44.041s

@jreback
Copy link
Contributor

jreback commented Feb 11, 2021

can you merge master and ping on green

@afeld afeld force-pushed the parallel-default branch 2 times, most recently from 02ea92f to 97c69ef Compare February 12, 2021 04:16
@afeld
Copy link
Member Author

afeld commented Feb 12, 2021

Anyone know what the NameError: name 'plt' is not defined is about?

@jreback
Copy link
Contributor

jreback commented Feb 12, 2021

Anyone know what the NameError: name 'plt' is not defined is about?

did you rebase on master? this should have gone away

@afeld
Copy link
Member Author

afeld commented Feb 13, 2021

Seems like the problem is still there.

$ git rev-parse HEAD
fc9fdba6592bdb5d0d1147ce4d65639acd897565
$ python make.py --single whatsnew/v0.8.0.rst
Running Sphinx v3.3.1
loading translations [en]... done
loading pickled environment... done
[autosummary] generating autosummary for: index.rst
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 1 source files that are out of date
updating environment: [config changed ('exclude_patterns')] 2 added, 0 changed, 1 removed
reading sources... [100%] whatsnew/v0.8.0                                                                    

>>>-------------------------------------------------------------------------
Exception in /root/workspace/pandas/doc/source/whatsnew/v0.8.0.rst at block ending on line 201
Specify :okexcept: as an option in the ipython:: block to suppress this message
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-2-65b822729fc4> in <module>
----> 1 plt.figure()

NameError: name 'plt' is not defined
<<<-------------------------------------------------------------------------
...

@jreback
Copy link
Contributor

jreback commented Feb 15, 2021

one more rebase if you pls

@jreback jreback added this to the 1.3 milestone Feb 15, 2021
Most people building docs would want that to happen as quickly as
possible, so this has Sphinx use the maximum parallelism
possible by default.
@jreback jreback merged commit 549ccb9 into pandas-dev:master Feb 17, 2021
@jreback
Copy link
Contributor

jreback commented Feb 17, 2021

thanks @afeld

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants