Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature]: remove useless file system "sync" for boltdb compaction #8502

Open
C-Otto opened this issue Feb 25, 2024 · 0 comments
Open

[feature]: remove useless file system "sync" for boltdb compaction #8502

C-Otto opened this issue Feb 25, 2024 · 0 comments
Labels
boltdb feature request Requests for new features P3 might get fixed, nice to have performance

Comments

@C-Otto
Copy link
Contributor

C-Otto commented Feb 25, 2024

Background

While the database compaction works, writing data to the "temp don't use" file, the updates are written in "sync" mode. For my HDD based ZFS system, this slows down operation quite a lot (it's not worth it to add a SLOG VDEV, as only lnd is using "sync"). Manually disabling "sync" on the filesystem (which one should not do in general!) massively improves the speed of the compaction process.

I believe it's not necessary to sync during compaction, as the data is only used once the whole process completes (running sync at the end might be necessary, though).

Your environment

  • 0.17.3-beta
  • Linux ftp 6.1.0-18-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.76-1 (2024-02-01) x86_64 GNU/Linux
  • v26

Steps to reproduce

Run bbolt compaction on, for example, the watchtower.db file of a watchtower (server).

Expected behaviour

It's fast.

Actual behaviour

It's slow.

@C-Otto C-Otto added bug Unintended code behaviour needs triage labels Feb 25, 2024
@C-Otto C-Otto changed the title [bug]: useless file system "sync" for boltdb compaction [feature]: remove useless file system "sync" for boltdb compaction Feb 25, 2024
@guggero guggero added feature request Requests for new features performance boltdb and removed bug Unintended code behaviour needs triage labels Feb 26, 2024
@saubyk saubyk added the P3 might get fixed, nice to have label Feb 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
boltdb feature request Requests for new features P3 might get fixed, nice to have performance
Projects
None yet
Development

No branches or pull requests

3 participants