Speed up enry.IsVendor

[zeripath]

enry.IsVendor is kinda slow as it simply iterates across all regexps.
This PR ajdusts the regexps to combine them to make this process a
little quicker.

Related #15143

Signed-off-by: Andrew Thornton art27@cantab.net

[zeripath]

This currently is about twice as fast as enry.IsVendor(...) on a non-vendored file

[silverwind]

Seems like a good idea. Linguist does similar, but they simply | the individual regex parts. Is this not an option here? Also, should upstream this.

[zeripath]

• Simply | the individual parts leads to a 4 times slow down.
• Sorting the regexps and | is 3 times slower.
• making the groups non capturing makes it twice as slow
• Placing all of the ^ first and making it common to the ones that use it is around equal
• Following those by (^|/) and with the common prefix extracted is 2 times faster.

This iterative process should suggest that some further simplifications should speed things up further. However if we drop full regexp compatibility we should be able to make it faster with a path trie. (I recently came across a library that looked reasonable for this - but its name escapes me.)

[zeripath]

My suspicion is that go's regexp simplifier is just not as good as ruby's or even that GitHub is using something like hyperscan to make it faster.

It's really worth noting that twice as fast here is still not really enough - it really would be better to be of the order of at least 10 times as fast.

Now in the associated issue the profile indicated more than a few other places that needed speed ups - however I suspect that some of these may be already on their way to being improved. (For example I think since the chi migration we shouldn't be listing branches or tags to do a http push anymore and we don't do a hash to assert if a password is set - we do however test password authentication twice during pushing!!)

[silverwind]

If enry.isVendor is a pure function (e.g. not reading files like .gitattributes), memoization could be another big gain if it's being called repeatedly on the same paths.

[zeripath]

I don't think we can assume that but I guess there are likely to be some common paths that could be captured. It's possible though that a trie can be just as fast as a cache.

Certainly though there are lots of places we could be memoizing by throwing stuff in to caches or precomputing - in particular commit verification.

[lafriks]

Maybe this could be contributed upstream?

[silverwind]

Should definitely upstream this, including the tests which already seem more extensive than enry's own. Thought I guess as a short-term solution it's fine.

[6543]

🚀