Evaluate Profile-Guided Optimization (PGO) and LLVM BOLT

[zamazan4ik]

Hi!

There are a lot of examples of different software, where Profile-Guided Optimization (PGO) helps with performance - you can check it here. E.g. in this list there are a lot of databases like PostgreSQL and ClickHouse.

We need to test PGO on pgvector.rs, and if it improves performance - at least write a note in the README file about it.

There are several additional options. I'd appreciate it if you could provide an easy way to build pgvector.rs with PGO. And experienced users will be able to do it on their own for their own usage scenarios. Another option is to optimize the pgvector.rs build with a generic-enough profile. Providing PGO-optimized binaries could be a trickier task (since it requires preparing a good-enough profile) but as an option would be great to see too. Probably completing the #21 task can help with it.

As an additional optimization way, I suggest taking a look at LLVM BOLT. But from my experience, it would be better to start with PGO and then try to use BOLT.

Treat this issue just like an idea for future improvements.

For the Rust projects, I recommend starting with cargo-pgo.

[gaocegege]

Thanks for the issue!

/cc @usamoi @VoVAllen

[workingjubilee]

This person has made many comments to this effect, however, I recommend not combining PGO and BOLT. It has caused many upstream issues for rust-lang, and Rust gets away with it mostly by having a lot of active contributors, such that when there's an all-hands-on-deck moment due to miscompilation, someone shows up to resolve it fairly quickly. It basically needs continuous monitoring to make sure nothing has gone wrong, because for whatever reason, rustc seems to be very good at generating code that surprises LLVM's tooling.

One or the other, independently, is probably safe to use, however. Rust had been using PGO for some time without BOLT, and it was only when used in tandem that they caused these issues.

[zamazan4ik]

Since I agree with your comments about BOLT and PGO stability issues, I still think PGO and BOLT combination is still a good thing to consider for the projects. Yes, combining PGO and BOLT can cause the issue but the decision on the tradeoff between "PGO+BOLT miscompilation possibilities" vs "gains from BOLT with PGO" should be considered for each project individually based on the projects' merits ("performance for all money", "stability first") and available resources like active contributors (as you said above).

Regarding BOLT + PGO issues - do you have a list of them anywhere near you? I am very interested in the story behind them and definitely will add to my PGO repository as the warnings for anyone who will test PGO with BOLT.

Thanks in advance!

[workingjubilee]

Absolutely, it could still be incredibly profitable for them if e.g. the team working on pgvecto.rs happened to have someone available to basically commit full time to making sure:

• PGO works
• BOLT works
• they don't introduce any problems
• maybe doing some other opportunistic performance opts
• and keeping an eye on that for a while after as their setup evolves. how long? ...I dunno!

Anyways, BOLT-related miscompiles:

• Miscompilation of is_whitespace inside rustc on Windows-msvc (with -Zdylib-lto) rust-lang/rust#109067
• 1.68.0 emits STATUS_ILLEGAL_INSTRUCTION on build rust-lang/rust#108978

Generic BOLT issues:

• rustc 1.66.0 requires execstack to load libLLVM.so rust-lang/rust#105783
• standalone installer rustc-1.70.0 (precompiled binary) segfaults in elf_machine_runtime_setup rust-lang/rust#112286
• Failed to compile a project with LTO + PGO rust-lang/rust#115344 the author of this seems familiar :^)

Relevant perf tracking issues for rustc:

• Tracking issue for speeding up rustc via its build configuration rust-lang/rust#103595
• Exploring PGO for the Rust compiler rust-lang/rust#79442

Rust partially mitigated this by extending testing to the optimized artifacts as well:

• Run tests for dist builds rust-lang/rust#109535

[zamazan4ik]

Thanks a lot for the links!

[Kobzol]

It has caused many upstream issues for rust-lang

I'm not aware of any miscompilations caused by BOLT so far. We had miscompilations caused by LTO on Windows, but we don't use BOLT there.

Maybe you meant LTO + PGO? But that just sometimes fails to compile, I haven't seen any miscompilations related to that.