Implementing mypyc support pt. 2

[ichard26]

It's finally happening 🎉, the initial work was done several years ago in GH-1009 by @ msullivan is now being followed through by yours truly. If this work doesn't find its way to PyPI eventually, I'll be disappointed 🙂

Performance

I wrote an entire report which can be read here: https://gist.github.com/ichard26/b996ccf410422b44fcd80fb158e05b0d, but the TL;DR is:

• Formatting with safety checks: 1.93x faster
• Formatting without safety checks: 2.11x faster
• Blib2to3 parsing: 1.81x faster
• Import (import black): 1.16x faster

Windows numbers are to come later. Unfortunately I probably won't be able to get MacOS numbers (I own no MacOS running machine), but I'll see what I can do.

Stability

I've been running the test suite during the wheel build and as of latest of this branch it's all green. Though it should be noted that the arm64 and the arm portion of the universial2 wheels are completely untested since we don't have access to any M1 CI resources :(

Locally I've ran diff-shades (Jelle, it's basically mypy-primer but for black) against the CPython 3.8 manylinux wheel and no difference between interpreted and compiled Black was found (across the same project suite used by black-primer + blackbench 😉). I plan to also test Windows with diff-shades in the following days. I'll also get Cooper to test Intel MacOS + M1 MacOS for me, but I need to come up with better instructions before that.

Anyhow, please don't expect this to be fully tested. I plan to ask the wider community to test the experimental compiled wheels (and push any additional fixes) after this PR is merged.

Review notes

This PR contains a lot of things including QOL improvements, optimizations, compatibility changes, hacks, etc. While the diff isn't that painful, following the commit history should give you some context on why certain changes were made.

It should be noted that I plan to handle the merge of this PR myself since this project involves a lot of moving parts and I'd like to maintain control for the time being. Thanks!

---

Related issue: GH-366
Related repo: https://github.com/ichard26/black-mypyc-wheels

And if you're some passerby who really wants to try the experimental wheels, checkout the workflows here: https://github.com/ichard26/black-mypyc-wheels/actions.

[cooperlees]

Nice work. FWIW - I'm totally down for black-primer to be refactored to do what it does today as well as what diff-shades does to cut down on the number of tools we all use everywhere. I guess diff-shades like behavior could be more easily added to our CI too if we incorporated it into the black repo.

[JelleZijlstra]

Thanks for all your work on this! I'm really excited to see this PR.

[JelleZijlstra]

Does this really help? re.compile already caches things internally I believe.

[ichard26]

Looking at the source and testing locally, I can confirm that both re and regex cache compiled patterns. Somehow I still observe a 8-9% speed up on that strings list literal file. Running this test script:

import time
from functools import lru_cache

import re
import regex

_cached_regex_compile = lru_cache(regex.compile)

# This is *roughly* how many times regex.compile was called formatting strings-list.py
loops = 410 
pattern = r"\s*\t+\s*(\S)"

re.compile(pattern)
regex.compile(pattern)

t0 = time.perf_counter()
for i in range(1, loops + 1):
    re.compile(pattern)
elapsed = time.perf_counter() - t0
print(f"{loops} cached compiles with re: {elapsed * 1000:.1f}ms")

t0 = time.perf_counter()
for i in range(1, loops + 1):
    regex.compile(pattern)
elapsed = time.perf_counter() - t0
print(f"{loops} cached compiles with regex: {elapsed * 1000:.1f}ms")

t0 = time.perf_counter()
for i in range(1, loops + 1):
    _cached_regex_compile(pattern)
elapsed = time.perf_counter() - t0
print(f"{loops} cached compiles with regex + lru_cache: {elapsed * 1000:.1f}ms")

I get:

• 410 cached compiles with re: 0.4ms
• 410 cached compiles with regex: 1.6ms
• 410 cached compiles with regex + lru_cache: 0.1ms

So it seems like this is the source of the slight speedup, but oddly enough this only accounts for ~half of the 8-9% speedup I saw for fmt-strings-list. Maybe the fact this script only repeatedly compiles one regex is why the delta isn't 1:1. For real life cases this speedup is probably minimal so I wouldn't mind reverting this to be less confusing :)

[JelleZijlstra]

Interesting, thanks for the extra information. I looked at re.compile and it calls a helper function (_compile) and does an isinstance call before looking at the cache, and the cache key is a tuple type(pattern), pattern, flags. I suppose that adds up to more overhead than whatever lru_cache does internally.

[ichard26]

Andddd it's broken now because of PR #2557 - https://github.com/ichard26/black-mypyc-wheels/runs/4040805225?check_suite_focus=true#step:5:197

[ichard26]

Still broken since the test suite is still dependent on CWD being something specific :(

https://github.com/ichard26/black-mypyc-wheels/runs/4040890632?check_suite_focus=true#step:5:811

gonna fix this later

[ichard26]

Whoops this isn't PyPy compatible ATM, will fix.