Code

Add support for memory-mapped files#604

Code

Merged

FrancescAlted merged 18 commits intoBlosc:mainBlosc/c-blosc2:mainfrom JanSellner:mmapJanSellner/c-blosc2:mmapCopy head branch name to clipboardMay 16, 2024

+1,102-187

Commits

Commits on May 2, 2024

• Change io_cb read API to allow passing back an existing pointer

  JanSellner

  committed
  230e8c8

  Copy full SHA for 230e8c8

• New IO mode using memory mapped files (no Windows support yet)

  JanSellner

  committed
  1c04f36

  Copy full SHA for 1c04f36

Commits on May 3, 2024

• Add Windows support for memory-mapped files

  JanSellner

  committed
  782b990

  Copy full SHA for 782b990

• fix warnings

  JanSellner

  committed
  56461f3

  Copy full SHA for 56461f3

• Ensure borders of memory-mapped file are respected

  JanSellner

  committed
  de42f28

  Copy full SHA for de42f28

• Use mapping size independent of file size and start with large enough mappings

  JanSellner

  committed
  c0f4336

  Copy full SHA for c0f4336

Commits on May 4, 2024

• Improve extended mapping size

  JanSellner

  committed
  d7278bc

  Copy full SHA for d7278bc

• New default initialization, added test for c mode

  JanSellner

  committed
  63c9389

  Copy full SHA for 63c9389

Commits on May 5, 2024

• Use different define to detect Windows systems

  JanSellner

  committed
  221af5f

  Copy full SHA for 221af5f

Commits on May 8, 2024

• Use different write file mapping mode on Windows which does not crash

  JanSellner

  committed
  7280ea6

  Copy full SHA for 7280ea6

• Introduce position argument to read/write functions instead of seeking to avoid multithreading issues

  JanSellner

  committed
  32eed17

  Copy full SHA for 32eed17

• Resolve review comments

  JanSellner

  committed
  5e1a292

  Copy full SHA for 5e1a292

Commits on May 13, 2024

• Remove seek and tell callbacks, introduce size instead

  JanSellner

  committed
  0c8787d

  Copy full SHA for 0c8787d

Commits on May 14, 2024

• Merge commit 'd2167d34120f617f64f48090f2495fc6e9ae8cf5' into mmap

  JanSellner

  committed
  694f0a1

  Copy full SHA for 694f0a1

• add function to retrieve default struct

  JanSellner

  committed
  115a697

  Copy full SHA for 115a697

• Add helper script for aggregated benchmarks across multiple runs

  JanSellner

  committed
  5d7e4e1

  Copy full SHA for 5d7e4e1

• Expand documentation and add a check that only one file is opened

  JanSellner

  committed
  cceb06d

  Copy full SHA for cceb06d

Commits on May 15, 2024

• Use msync only in the end

  JanSellner

  committed
  40e101b

  Copy full SHA for 40e101b